Revision history [back]

Ok let's answer this question. I've been digging into the opencv code, compiling all the necessary in debug mode and doing a step-by-step debuggin session of the same implementation I previously had in Python now in C++.

The code that deals with the projectpoints function is the calibration.cpp which in turns calls the cvProjectPoints2Internal routine. The projection itself is done in the lines around line 774, where a long for starts and this is the part where the K4 value (the 5th element of distortion) plays the key role:

    r2 = x*x + y*y;
    r4 = r2*r2;
    r6 = r4*r2;
    a1 = 2*x*y;
    a2 = r2 + 2*x*x;
    a3 = r2 + 2*y*y;
    cdist = 1 + k[0]*r2 + k[1]*r4 + k[4]*r6; <<<< ***HERE***
    icdist2 = 1./(1 + k[5]*r2 + k[6]*r4 + k[7]*r6);
    xd0 = x*cdist*icdist2 + k[2]*a1 + k[3]*a2 + k[8]*r2+k[9]*r4;
    yd0 = y*cdist*icdist2 + k[2]*a3 + k[3]*a1 + k[10]*r2+k[11]*r4;

My conclusion is that is not a real bug, it is simply the way in which the routine works. But, there's the issue of "returning" points, or whatever name we want to give to those points. The fact is that those points I put in the question are nothing less than these ones I've created using a much higher value for K4 (the 5th element of distorsion):

The issue than can be translated into "can we remove those points behind some kind of, let's say, distortion limit?"

I don't really know how to simply do that, maybe the jacobian evaluated for every point can be used, or maybe not, but certainly it would be an interesting extension of the current behavior of this function.

At this time, I do not have time to investigate for an elegant solution and my way to solve this issue was just to compare the position of the distorsion-free point (using all K's set to zero) and checking whether the point falls inside a specific region. I know, is not a real solution for the issue, is just a workaround for my specific problem with my cameras and blabla, but it works. Here is my C++ code with the solution for this temporary workaround, feel free to improve the solution :-D

Also, the fact that the points are projected even if they are behind the camera should be something to consider in the future versions of OpenCV... as also the definition of positive/negative K1 value, as I think the issue here is in the way we are thinking (3D to 2D or 2D to 3D...)

To conclude, here it is my workaround "in action" :-)

With the "problem" https://github.com/trigal/trigal.github.io/blob/master/images/opencv3d2dprojections/test6.gif

Without the "problem" https://github.com/trigal/trigal.github.io/blob/master/images/opencv3d2dprojections/test9.gif