solvePnPRansac on coplanar planes gives randomly one of two solutions
I have a problem when using solvePnPRansac to track coplanar markers. I recorded a video of the problem here:
In the video you see the pink cube drawn with OpenCV flipping about every 2-3 frames. You also see on top a stable coordinate system I render with OpenGL but you can ignore it, I don't apply the rotation values calculated by the tracker, that's why the flipping is not passed to the OpenGL scene.
I assume the problem lies in the solvePnPRansac of OpenCV since the inputs (the 2D and 3D points) are very similar on each frame but the calculated projection matrix differs a log as you can see in the video. The tracked points all look fine. In the video I also render the coordinate axes and you can see that the z-axis is flipped/inverted while the rest of the axes (Y-axis: green pointing down and X-axis: red point to the right) stay correct.
My used parameters for solvePnPRANSAC: I used cv::ITERATIVE on the solvePnPRANSAC with the standard reprojection error 8.0 (tried several times with different values but no luck) and I switched of the extrinsic guess to false.
Are there always 2 correct pnp solutions for coplanar markers and that's why it randomly flips between both? Can I avoid this behaviour. I searched for people having similar problems but all I could find about this are two old questions which sound like it could be the same problem: http://answers.opencv.org/question/63... and http://answers.opencv.org/question/11...
A reprojection error of 8 Pixels is rather high, could you try a smaller one? (How high is the reprojection errors for the matches features?) Could you create some images via drawMatches to show the final correspondences? How many inliers does the RANSAC return? What happens if you fix the matches and call solvePnPRansac several times on the same data? Does it also flip or is it stable? The coplanarity of the points is no problem as long as the matched points are not also on a line but spread across the whole plane. Could that happen in your case? Do you have the same effect with methods other than cv::iterative?
I've tried the reproj. err. = 0.006 * maxVal (I've read from some literature) but still got a flip. the inliers would be in the range from 60 to 80%.
I've run a small test, where I used the same 2d-3d points for solvePnPRansac, as the result I got the same R and t everytime, but also I got the flip as well.
Well, the tracked points look dense and spread on the whole plane... I still got a flip If I used another method say P3P or even EPNP, and the result even more unstable