OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Thu, 06 Sep 2018 11:07:22 -0500Difference between essential matrix computed using findEssentialMat and computed using (findFundamentalMat + calibration matrix)http://answers.opencv.org/question/198841/difference-between-essential-matrix-computed-using-findessentialmat-and-computed-using-findfundamentalmat-calibration-matrix/ Hello,
I am new to openCV and I developed a simple program aiming at computing the rotation and the translation between 2 successive camera frames (I use only one monocular camera). After matching the feature points, I computed the essential matrix using two different ways. The first method consists of applying the function findEseentialMat to the matched points and using the calibration matrix. The second method consists of applying findFundamentalMat to the mateched points and then applying the formula: E = K^T * F * K.
The obtained results of E are really different using the two methods and I cannot find an explanation to this.
Would you have an explanation, please?
Thank youAmaniThu, 06 Sep 2018 11:07:22 -0500http://answers.opencv.org/question/198841/How to compute the covariance of an inter-camera relative pose measurement?http://answers.opencv.org/question/179919/how-to-compute-the-covariance-of-an-inter-camera-relative-pose-measurement/If I'm doing pose estimation using a single camera using 3D-2D correspondences (E.g. PNP algorithm), I have read that reprojecting the points can give me an estimate of the Jacobian (cv::projectPoints), which can then be used to compute an estimate of the covariance of the pose.
But if I have two cameras, and I am performing relative pose estimation between the cameras using the fundamental/essential matrix (cv::findEssentialMat) and subsequent decomposition of the matrix, how can I compute the covariance of the relative pose between the cameras?saihvWed, 06 Dec 2017 20:23:32 -0600http://answers.opencv.org/question/179919/findEssentialMat() pose estimation: wrong translation vector in some caseshttp://answers.opencv.org/question/68328/findessentialmat-pose-estimation-wrong-translation-vector-in-some-cases/ I am currently using epipolar geometry based pose estimation for estimating pose of one camera w.r.t another, with non-zero baseline between the cameras. I am using the five-point algorithm (implemented as findEssentialMat in opencv) to determine the up-to-scale translation and rotation matrix between the two cameras.
I have found two interesting problems when working with this, it would be great if someone can share their views: I don't have a great theoretical background in computer vision:
1. If the rotation of the camera is along the Z axis i.e., parallel to the scene and the translation is non-zero, the translation between camera1 and camera2 (which is along X in the real world) is wrongly estimated to be along Z. Example case: cam1 and cam2 spaced by approx 0.5 m on the X axis, cam2 rotated clockwise by 45 deg.
Image pair
![image pair1][1]
Output:
Translation vector is [-0.02513, 0.0686, 0.9973] (wrong, should be along X)
Rotation Euler angles: [-7.71364, 6.0731, -43.7583] (correct)
2. The geometry between image1 and image2 is not the exact inverse of the geometry between image2 and image1. Again while an image1<=>image2 correspondence produces the correct translation, image2<=>image1 is way off (rotation values are close though). Example below, where camera2 was displaced along X and rotated for ~30 degrees along Y
Image 1 to image 2
![1to2][2]
Output: Rotation [-1.578, 24.94, -0.1631] (Close) Translation [-0.0404, 0.035, 0.998] (Wrong)
Image 2 to image 1
![2to1][3]
Output: Rotation [2.82943, -30.3206, -3.32636] Translation [0.99366, -0.0513, -0.0999] (Correct)
Looks like it has no issues figuring the rotations out but the translations are a hit or miss.
As to question 1, I was initially concerned because because the rotation is along the Z axis, the points might appear to be all coplanar. But the five point algorithm paper particularly states: "The 5-point method is essentially unaffected by the planar degeneracy and still works".
Thank you for your time!
[1]:http://s14.postimg.org/umezzqt7z/case1.jpg
[2]:http://s14.postimg.org/h5moo3wtr/1to2.png
[3]:http://s14.postimg.org/x55c7nsvj/2to1.pngsaihvMon, 10 Aug 2015 21:35:12 -0500http://answers.opencv.org/question/68328/Pose estimation: solvePnP and epipolar geometry do not agreehttp://answers.opencv.org/question/68149/pose-estimation-solvepnp-and-epipolar-geometry-do-not-agree/ Hi, I have a relative camera pose estimation problem where I am looking at a scene with differently oriented cameras spaced a certain distance apart. Initially, I am computing the essential matrix using the 5 point algorithm and decomposing it to get the R and t of camera 2 w.r.t camera 1.
I thought it would be a good idea to do a check by triangulating the two sets of image points into 3D, and then running solvePnP on the 3D-2D correspondences, but the result I get from solvePnP is way off. I am trying to do this because bundle adjustment would not work as my scale keeps changing; so I thought this would be one way to "refine" my pose: correct me if I am wrong. Anyway, In one case, I had a 45 degree rotation between camera 1 and camera 2 along the Z axis, and the epipolar geometry part gave me this answer:
Relative camera rotation is [1.46774, 4.28483, 40.4676]
Translation vector is [-0.778165583410928; -0.6242059242696293; -0.06946429947410336]
solvePnP, on the other hand..
Camera1: rvecs [0.3830144497209735; -0.5153903947692436; -0.001401186630803216]
tvecs [-1777.451836911453; -1097.111339375749; 3807.545406775675]
Euler1 [24.0615, -28.7139, -6.32776]
Camera2: rvecs [1407374883553280; 1337006420426752; 774194163884064.1] (!!)
tvecs[1.249151852575814; -4.060149502748567; -0.06899980661249146]
Euler2 [-122.805, -69.3934, 45.7056]
Something is troublingly off with the rvecs of camera2 and tvec of camera 1. My code involving the point triangulation and solvePnP looks like this:
points1.convertTo(points1, CV_32F);
points2.convertTo(points2, CV_32F);
// Homogenize image points
points1.col(0) = (points1.col(0) - pp.x) / focal;
points2.col(0) = (points2.col(0) - pp.x) / focal;
points1.col(1) = (points1.col(1) - pp.y) / focal;
points2.col(1) = (points2.col(1) - pp.y) / focal;
points1 = points1.t(); points2 = points2.t();
cv::triangulatePoints(P1, P2, points1, points2, points3DH);
cv::Mat points3D;
convertPointsFromHomogeneous(Mat(points3DH.t()).reshape(4, 1), points3D);
cv::solvePnP(points3D, points1.t(), K, noArray(), rvec1, tvec1, 1, CV_ITERATIVE );
cv::solvePnP(points3D, points2.t(), K, noArray(), rvec2, tvec2, 1, CV_ITERATIVE );
And then I am converting the rvecs through Rodrigues to get the Euler angles: but since rvecs and tvecs themselves seem to be wrong, I feel something's wrong with my process. Any pointers would be helpful. Thanks!
saihvFri, 07 Aug 2015 16:28:15 -0500http://answers.opencv.org/question/68149/Epipolar geometry pose estimation: Epipolar lines look good but wrong posehttp://answers.opencv.org/question/67540/epipolar-geometry-pose-estimation-epipolar-lines-look-good-but-wrong-pose/ I am trying to use OpenCV to estimate one pose of a camera relative to another, using SIFT feature tracking, FLANN matching and subsequent calculations of the fundamental and essential matrix. After decomposing the essential matrix, I check for degenerate configurations and obtain the "right" R and t.
Problem is, they never seem to be right. I am including a couple of image pairs:
1. Image 2 taken with 45 degree rotation along the Y axis and same position w.r.t. Image 1.
<a href="http://i.imgur.com/lEsdjFn.jpg">Image pair</a>
<a href="http://i.imgur.com/hCYV2kN.jpg">Result </a>
2. Image 2 taken from approx. couple of meters away along the negative X direction, slight displacement in the negative Y direction. Approx. 45-60 degree rotation in camera pose along Y axis.
<a href="http://i.imgur.com/zO1hwh3.jpg">Image pair</a>
<a href="http://i.imgur.com/nn803lk.jpg">Result</a>
The translation vector in the second case, seems to be overestimating the movement in Y and underestimating the movement in X. The rotation matrices when converted to Euler angles give wrong results in both the cases. This happens with a lot of other datasets as well. I have tried switching the fundamental matrix computation technique between RANSAC, LMEDS etc., and am now doing it with RANSAC and a second computation using only the inliers with the 8 point method. Changing the feature detection method does not help either. The epipolar lines seem to be proper, and the fundamental matrix satisfies x'.F.x = 0
Am I missing something fundamentally wrong here? Given the program understands the epipolar geometry properly, what could possibly be happening that results in a completely wrong pose? I am doing the check to make sure points lie in front of both cameras. Any thoughts/suggestions would be very helpful. Thanks!
<a href="http://pastebin.com/42PTHPP6">Code</a> for referencesaihvFri, 31 Jul 2015 13:35:56 -0500http://answers.opencv.org/question/67540/undistortPoints, findEssentialMat, recoverPose: What is the relation between their arguments?http://answers.opencv.org/question/65788/undistortpoints-findessentialmat-recoverpose-what-is-the-relation-between-their-arguments/**TL;DR**: What relation should hold between the arguments passed to `undistortPoints`, `findEssentialMat` and `recoverPose`.
I have code like the following in my program
Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients, noArray(), K);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients, noArray(), K);
Mat E = findEssentialMat(imgpts1, imgpts2, 1, Point2d(0,0), RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, 1.0, Point2d(0,0), mask);
I `undistort` the Points before finding the essential matrix. The doc states that one can pass the new camera matrix as the last argument. When omitted, points are in *normalized* coordinates (between -1 and 1). In that case, I would expect that I pass 1 for the focal length and (0,0) for the principal point to `findEssentialMat`, as the points are normalized. So I would think this to be the way:
1. **Possibility 1** (normalize coordinates)
Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients);
Mat E = findEssentialMat(imgpts1, imgpts2, 1.0, Point2d(0,0), RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, 1.0, Point2d(0,0), mask);
2. **Possibility 2** (do not normalize coordinates)
Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients, noArray(), K);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients, noArray(), K);
double focal = K.at<double>(0,0);
Point2d principalPoint(K.at<double>(0,2), K.at<double>(1,2));
Mat E = findEssentialMat(imgpts1, imgpts2, focal, principalPoint, RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, focal, principalPoint, mask);
However, I have found, that I only get reasonable results when I tell `undistortPoints` that the old camera matrix shall still be valid (I guess in that case only distortion is removed) and pass arguments to `findEssentialMat` as if the points were normalized, which they are not.
Is this a bug, insufficient documentation or user error?
**Update**
It might me that `correctedMatches` should be called with (non-normalised) image/pixel coordinates and the Fundamental Matrix, not E, this may be another mistake in my computation. It can be obtained by `F = K^-T * E * K^-1`themightyoarfishWed, 08 Jul 2015 05:43:24 -0500http://answers.opencv.org/question/65788/Estimate camera pose (extrinsic parameters) from homography / essential matrixhttp://answers.opencv.org/question/38340/estimate-camera-pose-extrinsic-parameters-from-homography-essential-matrix/I am trying to estimate the camera pose from an estimated homography as explained in chapter 9.6.2 of [Hartley & Zisserman's book](http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook2/HZepipolar.pdf).
Basically, it says the following:
Let `W=(0,-1,0; 1,0,0; 0,0,1)`. The left camera is assumed at `[I|0]`.
Given an SVD decomposition of the essential matrix `E`
SVD(E) = U*diag(1,1,0)*V'
the extrinsic camera parameters [R|t] of the right camera are one of the following four solutions:
[U W V' | U*(0,0,+1)']
[U W V' | U*(0,0,-1)']
[U W'V' | U*(0,0,+1)']
[U W'V' | U*(0,0,-1)']
Now I am struggling with a few main issues.
1) I only have access to an estimated homography `H`. The way I understand it, it's not exactly an essential matrix, if the singular values are not two equal values. Therefore, what I am struggling with is that instead of
SVD(H) = U * diag(1,1,0) * V'
the SVD decomposes into something like
SVD(H) = U * diag(70,1.6,0.0001) * W
It's really weird that the singular values are not almost identical and that large. My first question is why this happens and what to do about it? Normalization? Scaling?
2) After a lot of thinking and more experimenting, I came up with the following implementation:
static const Mat W = (Mat_<double>(3, 3) <<
1, 0, 0,
0, -1, 0,
0, 0, 1
);
// Compute SVD of H
SVD svd(H);
cv::Mat_<double> R1 = svd.u * Mat::diag(svd.w) * svd.vt;
cv::Mat_<double> R2 = svd.u * Mat::diag(svd.w) * W * svd.vt;
cv::Mat_<double> t = svd.u.col(2);
This way I get four possible solutions `[R1|t], [R1|-t], [R2|t], [R2|-t]`, which produce some sort of results.
Apparently, in W, I don't swap x/y coordinates and I don't invert the x-coordinate. Only the y-coordinate is inverted.
I believe the swap can be explained by different image coordinate systems. So column and row might be inverted in my implementation. But I can't explain why I only have to mirror and not rotate. And overall I am not sure, if the implementation is correct.
3) In theory I think I need to triangulate a pair of matches and determine whether the 3D point is front of both planes. Only one of the four solutions will satisfy that condition. However, I don't know how to determine the near plane's normal and distance from the uncalibrated projection matrix.
4) This is all used in the context of image stitching. My goal is to enhance the current image stitching pipeline to support not only cameras rotating around itself, but also translating cameras with little rotation. Some preliminary results and a follow-up question will be posted soon. (TODO)DuffycolaTue, 29 Jul 2014 12:50:57 -0500http://answers.opencv.org/question/38340/Pose estimation produces wrong translation vectorhttp://answers.opencv.org/question/18565/pose-estimation-produces-wrong-translation-vector/Hi,<br>
I'm trying to extract camera poses from a set of two images using features I extracted with BRISK. The feature points match quite brilliantly when I display them and the rotation matrix I get seems to be reasonable. The translation vector, however, is not.
I'm using the simple method of computing the fundamental matrix, essential matrix computing the SVD as presented in e.g. H&Z:
Mat fundamental_matrix =
findFundamentalMat(poi1, poi2, FM_RANSAC, deviation, 0.9, mask);
Mat essentialMatrix = calibrationMatrix.t() * fundamental_matrix * calibrationMatrix;
SVD decomp (essentialMatrix, SVD::FULL_UV);
Mat W = Mat::zeros(3, 3, CV_64F);
W.at<double>(0,1) = -1;
W.at<double>(1,0) = 1;
W.at<double>(2,2) = 1;
Mat R1= decomp.u * W * decomp.vt;
Mat R2= decomp.u * W.t() * decomp.vt;
if(determinant(R1) < 0)
R1 = -1 * R1;
if(determinant(R2) < 0)
R2 = -1 * R2;
Mat trans = decomp.u.col(2);
However, the resulting translation vector is horrible, especially the z coordinate: Usually it is near (0,0,1) regardless of the camera movement I performed while recording these images. Sometimes it seems that the first two coordinates might be kind of right, but they're far to small in comparison to the z coordinate (e.g. I moved the camera mainly in +x and the resulting vector is something like (0.2, 0, 0.98).
Any help would be appreciated.FiredragonwebSat, 10 Aug 2013 08:37:43 -0500http://answers.opencv.org/question/18565/