maym86's profile - activity

OpenCV version 3.4.2

I am taking a stereo pair and using recoverPose to get the [R|t] pose of the camera, If I start at the origin and use triangulatePoints the result looks somewhat like expected although I would have expected the z points to be positive;

These are the poses of the cameras [R|t]

p0: [1, 0, 0, 0;
     0, 1, 0, 0;
     0, 0, 1, 0]

P1: [0.9999726146107655, -0.0007533190856300971, -0.007362237354563941, 0.9999683127209806;
     0.0007569149205790131, 0.9999995956157767, 0.0004856419317479311, -0.001340876868928852;
     0.007361868534054914, -0.0004912012195572309, 0.9999727804360723, 0.007847012372698725]

I get these results where the red dot and the yellow line indicates the camera pose (x positive is right, y positive is down):

When I rotate the first camera by 58.31 degrees and then use recoverPose to get the relative pose of the second camera the results are wrong.

Pose matrices where P0 is rotated by 58.31 degrees around the y axis before calling my code below.

P0: [0.5253219888177297, 0, 0.8509035245341184, 0;
     0, 1, 0, 0;
     -0.8509035245341184, 0, 0.5253219888177297, 0]

P1: [0.5315721563840478, -0.0007533190856300971, 0.8470126770406503, 0.5319823932782873;
     -1.561037994149129e-05, 0.9999995956157767, 0.0008991799591322519, -0.001340876868928852;
     -0.8470130118915117, -0.0004912012195572309, 0.5315719296650566, -0.8467543535708145]

(x positive is right, y positive is down)

The pose of the second frame is calculated as follows:

new_frame->E = cv::findEssentialMat(last_frame->points, new_frame->points, K, cv::RANSAC, 0.999, 1.0, new_frame->mask);
int res = recoverPose(new_frame->E, last_frame->points, new_frame->points, K, new_frame->local_R, new_frame->local_t, new_frame->mask);
// https://stackoverflow.com/questions/37810218/is-the-recoverpose-function-in-opencv-is-left-handed 
// Convert so transformation is P0 -> P1
new_frame->local_t = -new_frame->local_t;
new_frame->local_R = new_frame->local_R.t();

new_frame->pose_t = last_frame->pose_t + (last_frame->pose_R * new_frame->local_t);
new_frame->pose_R = new_frame->local_R * last_frame->pose_R;
hconcat(new_frame->pose_R, new_frame->pose_t, new_frame->pose);

I then call triangulatePoints using the K * P0 and K * P1 on the corresponding points.

I feel like this is some kind of coordinate system issue as the points I would expect to have positive z values have a -z value in the plots and so the rotation is behaving strangely. I haven't been able to figure out what I need to do to fix it.

EDIT: Here is a gif of what's going on as I rotate through 360 degrees around Y. The cameras are still parallel. What am I missing, shouldn't the shape of the point cloud remain the same if both camera poses remain in relative positions even thought they have been rotated around the origin? Why are the points squashed into the X axis?

Solved it - This function convert the aggregated pose into a projection matrix that makes sense. inline cv::Mat getProj

Solved it - This function convert the aggregated pose into a projection matrix that makes sense. inline cv::Mat getProj

See comments for answer.

This only seems to be an issue when I rotate around the y and z axes. Rotation around the x axis looks ok.

Triangulation gives weird results for rotation OpenCV version 3.4.2 I am taking a stereo pair and using recoverPose to

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation I am taking a stereo pair and using recoverPose to get the [R|t] pose of

Triangulation gives weird results for rotation If I start at the origin and use triangulatePoints the result looks like

Triangulation gives weird results for rotation If I start at the origin and use triangulatePoints the result looks like

Filtering made all the difference. Once I removed all the points marked by the mask from recoverPose, the negative z poi

Filtering made all the difference. Once I removed all the points marked by the mask from recoverPose, the negative z poi

Filtering made all the difference. Once I removed all the points marked by the mask from recoverPose, the negative z poi

Filtering made all the difference. Once I removed all the points marked by the mask from recoverPose, removed the negati

Hello I am having some trouble with triangulatePoints and the resulting 3D points are coming out incorrectly. I am using Opencv 3.4.2 on ubuntu 16.04. This is the function that I use to triangulate from two vectors of cv::Poin2f containing matches (I have verified that these matches are correct and make sense between the images).

std::vector<cv::Point3d> triangulate(const cv::Point2f &pp, double focal, const std::vector<cv::Point2f> &points0, const std::vector<cv::Point2f> &points1, const cv::Mat &P0, const cv::Mat &P1){

    cv::Mat p_mat0(2, static_cast<int>(points0.size()), CV_64F);
    cv::Mat p_mat1(2, static_cast<int>(points1.size()), CV_64F);

    for (int i = 0; i < p_mat0.cols; i++) {
        p_mat0.at<double>(0, i) = (points0[i].x  - pp.x) / focal;
        p_mat0.at<double>(1, i) = (points0[i].y  - pp.y) / focal;
        p_mat1.at<double>(0, i) = (points1[i].x  - pp.x) / focal;
        p_mat1.at<double>(1, i) = (points1[i].y  - pp.y) / focal;
    }

    cv::Mat points_4d;
    cv::triangulatePoints(P0, P1, p_mat0, p_mat1, points_4d);
    std::vector<cv::Point3d> results;
    for (int i = 0; i < points4d.cols; i++) {
        results.emplace_back(cv::Point3d(points4d.at<double>(0, i) / points4d.at<double>(3, i),
                    points4d.at<double>(1, i) / points4d.at<double>(3, i),
                    points4d.at<double>(2, i) / points4d.at<double>(3, i)));
    }
    return results;
}

The input are the features tracked between two frames and the P0 and P1 are generated from findEssentialMatrix and recoverPose between the two sets of points. I know that points0 and points1 are reversed for the first two functions and switching them doesn't make a difference.

E = cv::findEssentialMat(points1, points0, focal_, pp_, cv::RANSAC, 0.999, 1.0, mask);
int res = recoverPose(E, points1, points0, R, t, focal_, pp_, mask);
cv::Mat P;
hconcat(R, t, P);
triangulate(pp, focal, points0, points1, cv::Mat::eye(3, 4, CV_64FC1), P);

The tracked features are as follows:

The resultant P mat for this example is as follows:

[0.999994351093655, 0.00216084437549141, -0.002574593630994021, -0.08315501607210679;
-0.002166127328107902, 0.9999955507641086, -0.002050937440673792, 0.02514760517749096;
0.002570150419386047, 0.002056502752743189, 0.9999945825469503, 0.9962192736821971];

The result looks like this where the plots at the bottom are the x and y and x and z coordinates of the result, centered on the red cross:

Plotted with this loop:

for (int j = 0; j < points_3d.size(); j++) {
     cv::Point2d draw_pos = cv::Point2d(points_3d[j].x * scale + drawXY.cols / 2,
                      points_3d[j].y * scale + drawXY.rows / 2);
     cv::circle(drawXY, draw_pos, 1, cv::Scalar(0, 255, 0), 1);
 }

Any idea what the issue is, I have been struggling with this for a few days.

Yeah I just figured it out right before seeing this. Filtering made all the difference. Once I removed all the points ma

Yeah I just figured it out right before seeing this. Filtering made all the difference. Once I removed all the points ma

Yeah I just figured it out right before seeing this. Filtering made all the difference. Once I removed all the points ma

Yeah I just figured it out right before seeing this. Filtering made all the difference. Once I removed all the points ma

recoverPose and triangulatePoints - 3D results are incorrect Hello I am having some trouble with triangulatePoints and t

recoverPose and triangulatePoints - 3D results are incorrect Hello I am having some trouble with triangulatePoints and t

recoverPose and triangulatePoints - 3D results are incorrect Hello I am having some trouble with triangulatePoints and t

recoverPose and triangulatePoints - 3D results are incorrect Hello I am having some trouble with triangulatePoints and t

recoverPose and triangulatePoints - 3D results are incorrect Hello I am having some trouble with triangulatePoints and t