# undistortPoints, findEssentialMat, recoverPose: What is the relation between their arguments?

TL;DR: What relation should hold between the arguments passed to undistortPoints, findEssentialMat and recoverPose.

I have code like the following in my program

     Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients, noArray(), K);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients, noArray(), K);

Mat E = findEssentialMat(imgpts1, imgpts2, 1, Point2d(0,0), RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, 1.0, Point2d(0,0), mask);


I undistort the Points before finding the essential matrix. The doc states that one can pass the new camera matrix as the last argument. When omitted, points are in normalized coordinates (between -1 and 1). In that case, I would expect that I pass 1 for the focal length and (0,0) for the principal point to findEssentialMat, as the points are normalized. So I would think this to be the way:

1. Possibility 1 (normalize coordinates)

 Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients);
Mat E = findEssentialMat(imgpts1, imgpts2, 1.0, Point2d(0,0), RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, 1.0, Point2d(0,0), mask);

2. Possibility 2 (do not normalize coordinates)

 Mat mask; // inlier mask
undistortPoints(imgpts1, imgpts1, K, dist_coefficients, noArray(), K);
undistortPoints(imgpts2, imgpts2, K, dist_coefficients, noArray(), K);
double focal = K.at<double>(0,0);
Point2d principalPoint(K.at<double>(0,2), K.at<double>(1,2));
Mat E = findEssentialMat(imgpts1, imgpts2, focal, principalPoint, RANSAC, 0.999, 3, mask);
correctMatches(E, imgpts1, imgpts2, imgpts1, imgpts2);
recoverPose(E, imgpts1, imgpts2, R, t, focal, principalPoint, mask);


However, I have found, that I only get reasonable results when I tell undistortPoints that the old camera matrix shall still be valid (I guess in that case only distortion is removed) and pass arguments to findEssentialMat as if the points were normalized, which they are not.

Is this a bug, insufficient documentation or user error?

Update

It might me that correctedMatches should be called with (non-normalised) image/pixel coordinates and the Fundamental Matrix, not E, this may be another mistake in my computation. It can be obtained by F = K^-T * E * K^-1

edit retag close merge delete

Sort by ยป oldest newest most voted

As it turns out, my data seemingly is off. By using manually labelled correspondences I determined that Possibility 1 and 2 are indeed the correct ones, as one would expect.

more

Official site

GitHub

Wiki

Documentation