# solvePnP and problems with perspective [closed]

I feed 8 corners of the cube, but after reprojection sometimes (not for all images) the nearest point becomes the farthest and vice versa.

Can anybody explain the reason and how to cope with it?

An example of correct case:

The code:

        float box_side_x = 6; //Centimetres
float box_side_y = 6;
float box_side_z = 6;
vector<Point3f> boxPoints;


//Fill the array of corners in object coordinates. x to right(view from camera), y down, z from camera.

        vector<Vec3d> boxCorners(8);
Vec3d boxCorner;
float x, y, z;
for (int h = 0; h < 2; ++h) {
for (int j = 0; j < 2; ++j) {
for (int i = 0; i < 2; ++i) {
x = box_side_x * i;
y = box_side_y * j;
z = box_side_z * h;
boxPoints.push_back(Point3f(x, y, z)); //For solvePnP()
boxCorners[i + 2 * j + 4 * h] = {x, y, z}; //For calculating output
}
}
}

solvePnP(boxPoints, pointBuf, cameraMatrix, distCoeffs, rvec, tvec, false);
Mat rmat;
Rodrigues(rvec, rmat);

Mat Result;
float S = 1;
for (int h = 0; h < 2; ++h) {
for (int j = 0; j < 2; ++j) {
for (int i = 0; i < 2; ++i) {
boxCorner = boxCorners[i+2*j+4*h]; //In centimetres
Result = S * (rmat * Mat(boxCorner) + tvec);
ObjPoints[i + 2 * j + 4 * h] = (Vec3f)Result; //In centimetres
}
}
}


In different file ObjPoints -> BoxCorners

    vector<float> X(8), Y(8), Z(8);
for (int i = 0; i < 8; i++) {
BoxCorner = BoxCorners[i]; //In centimetres
X[i] = K * (BoxCorner[0] + Lx);
Y[i] = K * (BoxCorner[1] + Ly);
Z[i] = K * (BoxCorner[2] + Lz);
}
Scalar color = CV_RGB(0, 0, 200), back_color = CV_RGB(0, 0, 100);
int thickness = 2;

namedWindow("Projection", WINDOW_AUTOSIZE);
Point pt1 = Point(0, 0), pt2 = Point(0, 0);
pt1 = Point(X[7], Y[7]), pt2 = Point(X[6], Y[6]);
line(drawing, pt1, pt2, back_color, thickness);
pt1 = Point(X[7], Y[7]); pt2 = Point(X[5], Y[5]);
line(drawing, pt1, pt2, back_color, thickness);
pt1 = Point(X[7], Y[7]); pt2 = Point(X[3], Y[3]);
line(drawing, pt1, pt2, back_color, thickness);

pt1 = Point(X[0], Y[0]), pt2 = Point(X[1], Y[1]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[0], Y[0]); pt2 = Point(X[2], Y[2]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[0], Y[0]), pt2 = Point(X[4], Y[4]);
line(drawing, pt1, pt2, color, thickness);

pt1 = Point(X[1], Y[1]); pt2 = Point(X[3], Y[3]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[3], Y[3]); pt2 = Point(X[2], Y[2]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[2], Y[2]); pt2 = Point(X[6], Y[6]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[6], Y[6]); pt2 = Point(X[4], Y[4]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[4], Y[4]); pt2 = Point(X[5], Y[5]);
line(drawing, pt1, pt2, color, thickness);
pt1 = Point(X[5], Y[5]); pt2 = Point(X[1], Y[1]);
line(drawing, pt1, pt2, color, thickness);
imshow("Projection", drawing);

edit retag reopen merge delete

### Closed for the following reason the question is answered, right answer was accepted by ya_ocv_user close date 2018-10-18 01:05:41.758080

1

Can you perhaps post the relevant portion of your code, and ideally, one image that causes it, and a very similar one that does not?

Also, check this bug report and see if it is the same: https://github.com/opencv/opencv/issues/8813

( 2018-10-16 18:26:33 -0500 )edit

Yes, it is something similar. solvePnP outputs the matrix which converts from object coordinates (attached to the front corner) to camera coordinates which I consider as follows. X goes horizontally from left to right as we see it, Y downwards. Then Z should go backwards behind the screen so Z coordinate of the front corner should be negative. In reality, it comes out negative or positive. Correct case (2 picture) - when it is positive. By the way, Z coordinate of the object points in the opposite direction - off the camera. X too in the opposite. Y - in the same, downwards. Relating my question to your link - solvePnP sometimes places the object not in front of the camera, but behind it (positive Z).

( 2018-10-17 10:38:01 -0500 )edit

Correction. There is some confusion about correspondence between camera coordinates and screen coordinates. As seen from the description what we see on screen is turned image on camera sensor. So indeed camera X-Y are the same that screen X-Y despite camera and user look in opposite directions. Hence camera Z should go towards the object and normally Z coordinate of the front corner should be positive. If the object is not rotated, all axes of camera and object coordinate systems point to the same directions.

( 2018-10-17 17:30:24 -0500 )edit

Sort by ยป oldest newest most voted

I add a constant positive shift Lz before output. This makes it possible to move lines into visible area in case they are out of the window. If initial Z1 and Z2 are negative and Z1 + Lz, Z2 + Lz are positive, the point which was closer to the coordinate origin (camera) becomes further away.

more

Official site

GitHub

Wiki

Documentation