Ask Your Question
0

solvePnP and problems with perspective [closed]

asked 2018-10-16 14:13:19 -0500

updated 2018-10-17 06:59:35 -0500

I feed 8 corners of the cube, but after reprojection sometimes (not for all images) the nearest point becomes the farthest and vice versa.

image description

Can anybody explain the reason and how to cope with it?

An example of correct case:

image description

The code:

        float box_side_x = 6; //Centimetres
        float box_side_y = 6;
        float box_side_z = 6;
        vector<Point3f> boxPoints;

//Fill the array of corners in object coordinates. x to right(view from camera), y down, z from camera.

        vector<Vec3d> boxCorners(8);
        Vec3d boxCorner;
        float x, y, z;
        for (int h = 0; h < 2; ++h) {
            for (int j = 0; j < 2; ++j) {
                for (int i = 0; i < 2; ++i) {
                    x = box_side_x * i;
                    y = box_side_y * j;
                    z = box_side_z * h;
                    boxPoints.push_back(Point3f(x, y, z)); //For solvePnP()
                    boxCorners[i + 2 * j + 4 * h] = {x, y, z}; //For calculating output
                }
            }
        }

        solvePnP(boxPoints, pointBuf, cameraMatrix, distCoeffs, rvec, tvec, false);
        Mat rmat;
        Rodrigues(rvec, rmat);

        Mat Result;
        float S = 1;
        for (int h = 0; h < 2; ++h) {
            for (int j = 0; j < 2; ++j) {
                for (int i = 0; i < 2; ++i) {
                    boxCorner = boxCorners[i+2*j+4*h]; //In centimetres
                    Result = S * (rmat * Mat(boxCorner) + tvec);
                    ObjPoints[i + 2 * j + 4 * h] = (Vec3f)Result; //In centimetres
                }
            }
        }

In different file ObjPoints -> BoxCorners

    vector<float> X(8), Y(8), Z(8);
    for (int i = 0; i < 8; i++) {
        BoxCorner = BoxCorners[i]; //In centimetres
        X[i] = K * (BoxCorner[0] + Lx);
        Y[i] = K * (BoxCorner[1] + Ly);
        Z[i] = K * (BoxCorner[2] + Lz);
    }
    Scalar color = CV_RGB(0, 0, 200), back_color = CV_RGB(0, 0, 100);
    int thickness = 2;

    namedWindow("Projection", WINDOW_AUTOSIZE);
    Point pt1 = Point(0, 0), pt2 = Point(0, 0);
        pt1 = Point(X[7], Y[7]), pt2 = Point(X[6], Y[6]);
        line(drawing, pt1, pt2, back_color, thickness);
        pt1 = Point(X[7], Y[7]); pt2 = Point(X[5], Y[5]);
        line(drawing, pt1, pt2, back_color, thickness);
        pt1 = Point(X[7], Y[7]); pt2 = Point(X[3], Y[3]);
        line(drawing, pt1, pt2, back_color, thickness);

        pt1 = Point(X[0], Y[0]), pt2 = Point(X[1], Y[1]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[0], Y[0]); pt2 = Point(X[2], Y[2]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[0], Y[0]), pt2 = Point(X[4], Y[4]);
        line(drawing, pt1, pt2, color, thickness);

        pt1 = Point(X[1], Y[1]); pt2 = Point(X[3], Y[3]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[3], Y[3]); pt2 = Point(X[2], Y[2]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[2], Y[2]); pt2 = Point(X[6], Y[6]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[6], Y[6]); pt2 = Point(X[4], Y[4]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[4], Y[4]); pt2 = Point(X[5], Y[5]);
        line(drawing, pt1, pt2, color, thickness);
        pt1 = Point(X[5], Y[5]); pt2 = Point(X[1], Y[1]);
        line(drawing, pt1, pt2, color, thickness);
imshow("Projection", drawing);
edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by ya_ocv_user
close date 2018-10-18 01:05:41.758080

Comments

1

Can you perhaps post the relevant portion of your code, and ideally, one image that causes it, and a very similar one that does not?

Also, check this bug report and see if it is the same: https://github.com/opencv/opencv/issues/8813

Tetragramm gravatar imageTetragramm ( 2018-10-16 18:26:33 -0500 )edit

Yes, it is something similar. solvePnP outputs the matrix which converts from object coordinates (attached to the front corner) to camera coordinates which I consider as follows. X goes horizontally from left to right as we see it, Y downwards. Then Z should go backwards behind the screen so Z coordinate of the front corner should be negative. In reality, it comes out negative or positive. Correct case (2 picture) - when it is positive. By the way, Z coordinate of the object points in the opposite direction - off the camera. X too in the opposite. Y - in the same, downwards. Relating my question to your link - solvePnP sometimes places the object not in front of the camera, but behind it (positive Z).

ya_ocv_user gravatar imageya_ocv_user ( 2018-10-17 10:38:01 -0500 )edit

Correction. There is some confusion about correspondence between camera coordinates and screen coordinates. As seen from the description what we see on screen is turned image on camera sensor. So indeed camera X-Y are the same that screen X-Y despite camera and user look in opposite directions. Hence camera Z should go towards the object and normally Z coordinate of the front corner should be positive. If the object is not rotated, all axes of camera and object coordinate systems point to the same directions.

ya_ocv_user gravatar imageya_ocv_user ( 2018-10-17 17:30:24 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2018-10-18 01:04:09 -0500

I add a constant positive shift Lz before output. This makes it possible to move lines into visible area in case they are out of the window. If initial Z1 and Z2 are negative and Z1 + Lz, Z2 + Lz are positive, the point which was closer to the coordinate origin (camera) becomes further away.

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2018-10-16 14:13:19 -0500

Seen: 78 times

Last updated: Oct 18 '18