# Rotating target changes distances computed with solvePnP

I've been working on finding the distance to a target (of known real-world dimensions). The distance seems consistent enough when viewed from a forward-facing angle, but when I rotate the target (without translating it) the distance changes: Here is the relevant code:

vector<Point3f> objectPoints;
objectPoints.push_back(Point3f(-1, -1, 0));
objectPoints.push_back(Point3f(-1, 1, 0));
objectPoints.push_back(Point3f(1, 1, 0));
objectPoints.push_back(Point3f(1, -1, 0));
Mat objectPointsMat(objectPoints);

Mat rvec;
Mat tvec;
solvePnP(objectPointsMat, inputContour, cameraMatrix, distortionCoeffs, rvec, tvec);

double tx = tvec.at<double>(0, 0);
double ty = tvec.at<double>(1, 0);
double tz = tvec.at<double>(2, 0);
double dist = std::sqrt(tx*tx + ty*ty + tz*tz);

char *str;
asprintf(&str, "DIST = %7.3f", dist);
putText(original, str, Point(0, original.size().height), CV_FONT_HERSHEY_PLAIN,
3, Scalar(255, 255, 255), 3);


EDIT: As Eduardo suggested, I have drawn the computed prose. I have also changed the objects points to match the width/height ratio of the actual target using:

objectPoints.push_back(Point3f(-1, -0.7, 0));
objectPoints.push_back(Point3f(-1, 0.7, 0));
objectPoints.push_back(Point3f(1, 0.7, 0));
objectPoints.push_back(Point3f(1, -0.7, 0));


The issue is that while yawing seems to work now, rolling makes the distance jump: edit retag close merge delete

Maybe you could try to display the object frame to see where the computed pose is reprojected in the image. An example of how to do this here and here.

You could also check if the correspondences 3D object points and 2D image points are correct, check if the cameraMatrix and the distortion coefficients are correct.

What units are you using? The width of the rect is 2 but it's not meters. How did you specify the distance of the features during the intrinsic calibration? Could you also show the reprojection error? (Just apply cv::projectPoints to the 3d points and compare them to the features you found in the image)

Looking at the rotation vector you rendered in the second image it appears to shift the normal from toward the screen to straight up. I am also working on the same problem (although in python) and noticing the same issue even when describing the whole U shape rather than the bounding corners.

I can't find a way to send a private message, but can I pick your code on how you were able to detect this rectangle? I'm trying to do something similar with a hexagon shape, but I'm having problems.

Sort by » oldest newest most voted

The ordering of your extracted points is jumping. If the rect is in the left part of the image, the green axis is pointing right out of the plane and everything looks ok. In other cases, the green and red axis look as if the both have an angle of 45 deg to your pattern. In this situation, solvePnP thinks that the pattern is held so that the longer side is held vertical but tilted.

Could you add labels for the points in inputContour (cv::puttext)? If you assign the 2d and 3d points incorrectly, the pose will just flip.

In one configuration, the height of the object is assumed to be 1.4 at a distance of about 5.4, in the other the height is assumed to be 2 at a distance of about 7.2. 7.2/5.4 ~ 2/1.4 which is also a hint that this is really the problem.

However, the units really look strange.

more

I think the unit is in foot ?

The contour points are displayed in this gif: http://i.imgur.com/Au6tDQ4.gif Is there a method for consistently ordering the points?

I've also changed the object points to match the size of the real-life target in inches.

@augustt198 Ran into the same issue here, took my first point (which is assumed to be the top left or top right from the contour detection) and the centroid of the contours. If the first point fell to the right of the centroid you need to flip the array so you start at the top left.

Next, I looked at the second and last point to determine if the points were going clockwise or counterclockwise. If the y of the last point was less than the second point then the order needs to be swapped retaining the first point.

After both were done the points were ordered from top left and going clockwise. Also note that I defined all 8 points not just the outer 4 corners.

Alternatively you can detect these two cases and pass in altered versions of the target descriptor to compensate

Official site

GitHub

Wiki

Documentation