# Is anyone using P3P or EPNP in solvePnP?

Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.

Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.

Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.

As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?)

Thanks, Alex

--- Repro case:

Hi Alexander,

What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you.

My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here http://www.uco.es/investiga/grupos/ava/node/26), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.

Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this

vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);

Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}


and then update the call to solvePnP as

cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);


I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around.

Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.)

If you'd ...

edit retag close merge delete

Is there any documentation or references that can explain how the joint rotation-translation matrix [R|t] is computed once the position of the 3-D points are found in camera coordinates using the EPnP algorithm? I have found the EPnP paper (http://cvlabwww.epfl.ch/~lepetit/papers/lepetit_ijcv08.pdf) to be helpful in understanding how the 3-D locations are found in camera coordinates, but the paper does not explain how [R|t] is determined.

Thank you to anyone who can point me in the right direction. I am working on my Master's thesis and need to understand how [R|t] is found so that I can write about it.

( 2013-07-23 12:36:37 -0500 )edit

I can confirm the problem (OpenCV 2.4.6). solvePnP with the iterative algorithm works fine and gives me brilliant results. EPnP however is very unstable and occasionally gives completely wrong camera poses. Unusable for me. I suspect it's an implementation error, because EPnP is so famous. It should be better than this.

( 2013-12-13 06:42:57 -0500 )edit

How good were your results. I am using the iterative algrorithem on Android, which turned out to be the closet of all three. Anyways my results are far from brilliant. Rotation is sometimes confused with translation. I have +-5° error in the rotation and sometimes I experience outliners with 30° drift!

( 2015-03-24 13:24:57 -0500 )edit

Sort by » oldest newest most voted

Hello, Alex!

I added these algorithms to the OpenCV. Also I am using them in my projects (but with Ransac scheme). If you can give me some data for solvePnP, that leads to incorrect result, I try to investigate it.

more

Official site

GitHub

Wiki

Documentation