OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Tue, 24 Mar 2015 13:24:57 -0500Is anyone using P3P or EPNP in solvePnP?http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.
Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.
Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.
As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?)
Thanks,
Alex
--- Repro case:
Hi Alexander,
What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you.
My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here http://www.uco.es/investiga/grupos/ava/node/26), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.
Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this
vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
and then update the call to solvePnP as
cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around.
Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.)
If you'd like me to grab a bunch of data from bad frames and give you that then I can do, but that's the test harness that I just checked with just now. I don't see any changes to the relevant files recently in SVN, so assume 2.4.2 will have the same issues as 2.4.1.
--
PS. Thanks for creating this answers interface - I hope it works as well as other sites as the yahoo group was unusable. I believe I'd already posted this question there but it never actually showed up!Wed, 11 Jul 2012 13:30:11 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/Comment by FndrPlayer39 for <div class="snippet"><p>Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.</p>
<p>Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.</p>
<p>Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.</p>
<p>As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?) </p>
<p>Thanks,
Alex</p>
<p>--- Repro case: </p>
<p>Hi Alexander,</p>
<p>What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you. </p>
<p>My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here <a href="http://www.uco.es/investiga/grupos/ava/node/26">http://www.uco.es/investiga/grupos/ava/node/26</a>), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.</p>
<p>Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this</p>
<pre><code>vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
</code></pre>
<p>and then update the call to solvePnP as</p>
<pre><code>cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
</code></pre>
<p>I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around. </p>
<p>Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.) </p>
<p>If you'd ...<span class="expander"> <a>(more)</a></span></p></div>http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=17361#post-id-17361Is there any documentation or references that can explain how the joint rotation-translation matrix [R|t] is computed once the position of the 3-D points are found in camera coordinates using the EPnP algorithm? I have found the EPnP paper (http://cvlabwww.epfl.ch/~lepetit/papers/lepetit_ijcv08.pdf) to be helpful in understanding how the 3-D locations are found in camera coordinates, but the paper does not explain how [R|t] is determined.
Thank you to anyone who can point me in the right direction. I am working on my Master's thesis and need to understand how [R|t] is found so that I can write about it.Tue, 23 Jul 2013 12:36:37 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=17361#post-id-17361Comment by TokaM for <div class="snippet"><p>Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.</p>
<p>Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.</p>
<p>Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.</p>
<p>As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?) </p>
<p>Thanks,
Alex</p>
<p>--- Repro case: </p>
<p>Hi Alexander,</p>
<p>What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you. </p>
<p>My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here <a href="http://www.uco.es/investiga/grupos/ava/node/26">http://www.uco.es/investiga/grupos/ava/node/26</a>), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.</p>
<p>Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this</p>
<pre><code>vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
</code></pre>
<p>and then update the call to solvePnP as</p>
<pre><code>cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
</code></pre>
<p>I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around. </p>
<p>Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.) </p>
<p>If you'd ...<span class="expander"> <a>(more)</a></span></p></div>http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=58250#post-id-58250How good were your results. I am using the iterative algrorithem on Android, which turned out to be the closet of all three. Anyways my results are far from brilliant. Rotation is sometimes confused with translation. I have +-5° error in the rotation and sometimes I experience outliners with 30° drift!Tue, 24 Mar 2015 13:24:57 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=58250#post-id-58250Comment by David Scherfgen for <div class="snippet"><p>Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.</p>
<p>Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.</p>
<p>Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.</p>
<p>As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?) </p>
<p>Thanks,
Alex</p>
<p>--- Repro case: </p>
<p>Hi Alexander,</p>
<p>What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you. </p>
<p>My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here <a href="http://www.uco.es/investiga/grupos/ava/node/26">http://www.uco.es/investiga/grupos/ava/node/26</a>), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.</p>
<p>Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this</p>
<pre><code>vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
</code></pre>
<p>and then update the call to solvePnP as</p>
<pre><code>cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
</code></pre>
<p>I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around. </p>
<p>Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.) </p>
<p>If you'd ...<span class="expander"> <a>(more)</a></span></p></div>http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=25412#post-id-25412I can confirm the problem (OpenCV 2.4.6). solvePnP with the iterative algorithm works fine and gives me brilliant results. EPnP however is very unstable and occasionally gives completely wrong camera poses. Unusable for me. I suspect it's an implementation error, because EPnP is so famous. It should be better than this.Fri, 13 Dec 2013 06:42:57 -0600http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=25412#post-id-25412Answer by Tran Thu Do for <div class="snippet"><p>Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.</p>
<p>Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.</p>
<p>Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.</p>
<p>As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?) </p>
<p>Thanks,
Alex</p>
<p>--- Repro case: </p>
<p>Hi Alexander,</p>
<p>What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you. </p>
<p>My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here <a href="http://www.uco.es/investiga/grupos/ava/node/26">http://www.uco.es/investiga/grupos/ava/node/26</a>), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.</p>
<p>Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this</p>
<pre><code>vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
</code></pre>
<p>and then update the call to solvePnP as</p>
<pre><code>cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
</code></pre>
<p>I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around. </p>
<p>Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.) </p>
<p>If you'd ...<span class="expander"> <a>(more)</a></span></p></div> http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?answer=1226#post-id-1226I also want to use the solvePnP, because I dont' have the exact matches, so I am trying to use the solvePnpRansac, however the result often returns 0,0,0 or very large with ITERATIVE method for both rotation and translation. I don't know what's wrong with my code.
Sometimes the results change frequently eventhough I do not move my object. I have try to
----------
**ITERATIVE RESULT**
--------------------
TIME TO SOLVE PNP problem = 132.189926 ms
T = [-2795246.726793,-2795234.084584,2795231.968800]; R =[-0.788772, -0.799549, -2.217896]
Number of matches = 40; Number of current keypoints = 323
TIME TO SOLVE PNP problem = 131.272075 ms
T = [-2904305.899281,-2904293.059600,2904290.988266]; R =[3.005587, 1.634340, 0.464340]Number of matches = 40; Number of current keypoints = 301
TIME TO SOLVE PNP problem = 144.615681 ms
T = [-2647890.943078,-2647887.363847,2647881.109238];
R =[-0.331646, -0.027772, -0.594331]
Number of matches = 40;
Number of current keypoints = 319
**P3P RESULT**
--------------
TIME TO SOLVE PNP problem = 9.689402 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 298
TIME TO SOLVE PNP problem = 10.904786 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 322
TIME TO SOLVE PNP problem = 10.295081 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 320
TIME TO SOLVE PNP problem = 11.750907 ms
T = [4.711286,0.045493,-1.881079]; R =[1.717873, -1.533295, 0.630865]Number of matches = 40; Number of current keypoints = 322
TIME TO SOLVE PNP problem = 10.207615 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 327
TIME TO SOLVE PNP problem = 11.458132 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 310
TIME TO SOLVE PNP problem = 11.312476 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 319
TIME TO SOLVE PNP problem = 10.350709 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 312
TIME TO SOLVE PNP problem = 8.923428 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 315
TIME TO SOLVE PNP problem = 14.019550 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 318
TIME TO SOLVE PNP problem = 9.903494 ms
T = [-0.468120,4.181766,2.396880]; R =[-0.820046, 0.223524, -2.960140]Number of matches = 40; Number of current keypoints = 306
TIME TO SOLVE PNP problem = 12.618254 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 302
TIME TO SOLVE PNP problem = 10.571754 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 310
TIME TO SOLVE PNP problem = 9.754545 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 310
TIME TO SOLVE PNP problem = 10.931868 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 309
TIME TO SOLVE PNP problem = 10.254825 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 291
TIME TO SOLVE PNP problem = 11.041293 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 323
TIME TO SOLVE PNP problem = 11.661976 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 300
TIME TO SOLVE PNP problem = 8.481338 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 300
TIME TO SOLVE PNP problem = 9.653903 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 320
TIME TO SOLVE PNP problem = 9.421147 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 310
TIME TO SOLVE PNP problem = 10.598470 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 305
TIME TO SOLVE PNP problem = 11.956948 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 311
TIME TO SOLVE PNP problem = 11.657585 ms
T = [3.997804,-5.120469,1.958653]; R =[0.769224, -2.073554, 0.871845]Number of matches = 40; Number of current keypoints = 304
TIME TO SOLVE PNP problem = 11.101677 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 298
TIME TO SOLVE PNP problem = 10.807438 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 316
TIME TO SOLVE PNP problem = 13.159523 ms
T = [2.212497,1.368370,2.570467]; R =[1.430678, -0.850851, 2.114037]Number of matches = 40; Number of current keypoints = 326
TIME TO SOLVE PNP problem = 13.058515 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 304
TIME TO SOLVE PNP problem = 10.693622 ms
T = [3.991786,-0.258174,2.646804]; R =[1.424127, -1.432924, 1.628038]Number of matches = 40; Number of current keypoints = 313
TIME TO SOLVE PNP problem = 13.006182 ms
T = [3.991786,-0.258174,2.646804]; R =[1.424127, -1.432924, 1.628038]Number of matches = 40; Number of current keypoints = 313
TIME TO SOLVE PNP problem = 12.374519 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 307
TIME TO SOLVE PNP problem = 9.434688 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 321
TIME TO SOLVE PNP problem = 12.734266 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 307
TIME TO SOLVE PNP problem = 11.762984 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 320
TIME TO SOLVE PNP problem = 10.813660 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 312
**EPNP method**
---------------
TIME TO SOLVE PNP problem = 19.396737 ms
T = [-3072673.308893,-3072665.021231,3072663.227400]; R =[-2.206905, -1.514715, 0.572620]Number of matches = 40; Number of current keypoints = 326
TIME TO SOLVE PNP problem = 17.763050 ms
T = [-4430498.990264,-4430502.384659,4430490.715347]; R =[-0.262988, -0.053750, -0.025270]Number of matches = 40; Number of current keypoints = 319
TIME TO SOLVE PNP problem = 21.748089 ms
T = [-4789222.253857,-4789216.489710,4789213.483485]; R =[-1.683341, -0.728048, 0.428769]Number of matches = 40; Number of current keypoints = 309
TIME TO SOLVE PNP problem = 19.544954 ms
T = [-215646923531.820370,-215646769282.541630,215646400816.189060]; R =[-0.687943, -0.308430, -0.425654]Number of matches = 40; Number of current keypoints = 312
TIME TO SOLVE PNP problem = 25.813642 ms
T = [-4750463.085425,-4750458.735488,4750455.011046]; R =[1.298622, -0.478069, 2.383880]Number of matches = 40; Number of current keypoints = 307
TIME TO SOLVE PNP problem = 25.916113 ms
T = [-3259915.370281,-3259907.331363,3259907.043971]; R =[-1.856934, -0.650150, -0.595831]Number of matches = 40; Number of current keypoints = 308
TIME TO SOLVE PNP problem = 21.710760 ms
T = [-2778918.678748,-2778911.642014,2778906.275713]; R =[0.333012, -1.122143, -2.192609]Number of matches = 40; Number of current keypoints = 334
TIME TO SOLVE PNP problem = 21.342595 ms
T = [-63127576270.071701,-63127523594.673889,63127416366.945328]; R =[-1.744601, 0.375079, 2.407424]Number of matches = 40; Number of current keypoints = 324
TIME TO SOLVE PNP problem = 14.051756 ms
T = [-8031980.403052,-8031975.888226,8031958.422819]; R =[-0.698578, -1.027018, -0.244631]Number of matches = 40; Number of current keypoints = 322
TIME TO SOLVE PNP problem = 18.395811 ms
T = [-22285625289.183907,-22285610177.663429,22285564520.516361]; R =[-1.246430, -0.872454, 0.019194]Number of matches = 40; Number of current keypoints = 318
TIME TO SOLVE PNP problem = 29.550920 ms
T = [-141685931136.099730,-141685900732.077030,141685627113.505190]; R =[-2.035022, 1.124255, 1.824528]Number of matches = 40; Number of current keypoints = 331
TIME TO SOLVE PNP problem = 15.611883 ms
T = [-17564129000.391254,-17564120833.940914,17564089426.651958]; R =[-0.371025, -0.135765, -0.938731]Number of matches = 40; Number of current keypoints = 300
TIME TO SOLVE PNP problem = 16.092034 ms
T = [-3381862.279371,-3381860.848704,3381853.981334]; R =[2.256818, -0.474964, 0.832250]Number of matches = 40; Number of current keypoints = 313
TIME TO SOLVE PNP problem = 29.208739 ms
T = [-4302254.059587,-4302248.205839,4302246.433171]; R =[-1.760113, -0.777269, 0.457448]Number of matches = 40; Number of current keypoints = 332
TIME TO SOLVE PNP problem = 16.549496 ms
T = [-2881071.997986,-2881062.868242,2881060.088298]; R =[-2.614361, -1.548243, 0.492757]Number of matches = 40; Number of current keypoints = 308
TIME TO SOLVE PNP problem = 22.197499 ms
T = [-12541555351.501265,-12541547868.559233,12541524697.637314]; R =[-1.328401, -0.915568, -1.621294]Number of matches = 40; Number of current keypoints = 307
TIME TO SOLVE PNP problem = 17.894799 ms
T = [-3130396.218219,-3130387.493222,3130381.692337]; R =[2.483759, 1.371196, -0.212239]Number of matches = 40; Number of current keypoints = 313
TIME TO SOLVE PNP problem = 15.649577 ms
T = [-3531553.872714,-3531545.042227,3531539.897130]; R =[2.513603, 1.625164, -0.402303]Number of matches = 40; Number of current keypoints = 327
TIME TO SOLVE PNP problem = 27.336074 ms
T = [-6492669442.860683,-6492665149.189883,6492652828.782601]; R =[-1.373793, -0.749381, -1.371790]Number of matches = 40; Number of current keypoints = 323
TIME TO SOLVE PNP problem = 25.473290 ms
T = [-2700499.397379,-2700490.795146,2700486.126411]; R =[-1.931443, -1.649979, -1.114408]Number of matches = 40; Number of current keypoints = 314
TIME TO SOLVE PNP problem = 15.776569 ms
T = [-5389233.997580,-5389226.157772,5389225.652378]; R =[-2.282103, -0.528469, 0.586103]Number of matches = 40; Number of current keypoints = 318
TIME TO SOLVE PNP problem = 16.477766 ms
T = [-891905219935.799440,-891904583884.118040,891902577175.132200]; R =[3.074886, 0.047160, -0.362315]Number of matches = 40; Number of current keypoints = 299
TIME TO SOLVE PNP problem = 26.246583 ms
T = [-21862112337.236137,-21862103296.517002,21862060041.655159]; R =[-2.133291, -0.519164, -0.049180]Number of matches = 40; Number of current keypoints = 303
TIME TO SOLVE PNP problem = 18.424356 ms
T = [-3830448.335440,-3830440.813513,3830431.202470]; R =[0.207558, -1.022248, -1.820957]Number of matches = 40; Number of current keypoints = 305
TIME TO SOLVE PNP problem = 21.319173 ms
T = [-3310586.913832,-3310577.368542,3310575.455972]; R =[-2.400846, -1.277314, 0.535444]Number of matches = 40; Number of current keypoints = 298
TIME TO SOLVE PNP problem = 24.419299 ms
T = [-3655016.057833,-3655009.615712,3655007.852158]; R =[-1.813108, -1.199822, 0.490473]Number of matches = 40; Number of current keypoints = 314
Wed, 08 Aug 2012 00:01:47 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?answer=1226#post-id-1226Comment by Kirill Kornyakov for <div class="snippet"><p>I also want to use the solvePnP, because I dont' have the exact matches, so I am trying to use the solvePnpRansac, however the result often returns 0,0,0 or very large with ITERATIVE method for both rotation and translation. I don't know what's wrong with my code.</p>
<p>Sometimes the results change frequently eventhough I do not move my object. I have try to </p>
<hr>
<h2><strong>ITERATIVE RESULT</strong></h2>
<pre><code>TIME TO SOLVE PNP problem = 132.189926 ms
T = [-2795246.726793,-2795234.084584,2795231.968800]; R =[-0.788772, -0.799549, -2.217896]
Number of matches = 40; Number of current keypoints = 323
TIME TO SOLVE PNP problem = 131.272075 ms
T = [-2904305.899281,-2904293.059600,2904290.988266]; R =[3.005587, 1.634340, 0.464340]Number of matches = 40; Number of current keypoints = 301
TIME TO SOLVE PNP problem = 144.615681 ms
T = [-2647890.943078,-2647887.363847,2647881.109238];
R =[-0.331646, -0.027772, -0.594331]
Number of matches = 40;
Number of current keypoints = 319
</code></pre>
<h2><strong>P3P RESULT</strong></h2>
<pre><code>TIME TO SOLVE PNP problem = 9.689402 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 298
TIME TO SOLVE PNP problem = 10.904786 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 322
TIME TO SOLVE PNP problem = 10.295081 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 320
TIME TO SOLVE PNP problem = 11.750907 ms
T = [4.711286,0.045493,-1.881079]; R =[1.717873, -1.533295, 0.630865]Number of matches = 40; Number of current keypoints = 322
TIME TO SOLVE PNP problem = 10.207615 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 327
TIME TO SOLVE PNP problem = 11.458132 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 310
TIME TO SOLVE PNP problem = 11.312476 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 319
TIME TO SOLVE PNP problem = 10.350709 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 312
TIME TO SOLVE PNP problem = 8.923428 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 315
TIME TO SOLVE PNP problem = 14.019550 ms
T = [0.000000,0.000000,0.000000]; R =[0.000000, 0.000000, 0.000000]Number of matches = 40; Number of current keypoints = 318
TIME TO SOLVE PNP problem = 9.903494 ms
T = [-0.468120,4.181766,2.396880]; R =[-0.820046, 0.223524, -2.960140]Number of matches = 40 ...</code></pre><span class="expander"> <a>(more)</a></span></div>http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=1297#post-id-1297Please create a new question.Thu, 09 Aug 2012 01:55:35 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?comment=1297#post-id-1297Answer by AlexanderShishkov for <div class="snippet"><p>Is anyone using solvePnP with the new solvers (P3P or EPNP?) I would like to explore doing so, primarily as a performance boost, but I get unusable results when doing so.</p>
<p>Specifically, EPNP seems to randomly return orientations that are 90 degrees off (i.e. the x-axis points along where the y-axis should be.) P3P seems to be extremely unstable to inaccuracies in the points (even though I'm passing it heavily refined corners of a known flat square) and returns garbage as a result.</p>
<p>Is anyone else using these algorithms in production code? I saw in the P3P code comments saying that several edge cases were not yet implemented. Are there plans to make these implementations efficient and/or robust, or should I just assume the ITERATIVE method is the only viable solution now.</p>
<p>As a follow-up, assuming that I am interested in performance optimization, is there anything I can do to speed up solving with CV_ITERATIVE? In particular, has anyone replaced OpenCV's LevMarq solver with a solver from a different package (is it possible to use Apple's accelerate framework for example?) </p>
<p>Thanks,
Alex</p>
<p>--- Repro case: </p>
<p>Hi Alexander,</p>
<p>What would you like to help solve the problem? I can print out some vectors of coefficients from frames that fail and post them here, or I can describe how to make code to reproduce the issues, which is likely more helpful for you. </p>
<p>My test case for P3P is very easy - download the Aruco library (I just tested using 1.2.4 from here <a href="http://www.uco.es/investiga/grupos/ava/node/26">http://www.uco.es/investiga/grupos/ava/node/26</a>), and run the aruco_test sample with a generated fiducial marker. At runtime when the marker is visible on the camera, this will pass 4 points that are refined with reasonable confidence to solvePnP in marker.cpp.</p>
<p>Passing Mat parameters didn't work for me with OpenCV 2.4.1, so I copied them in to std::vectors and pass those instead. i.e, in marker.cpp, so immediately above the call to solvePnP, paste in this</p>
<pre><code>vector<Point2f> imgPointsVector;
vector<Point3f> objPointsVector;
for (int c = 0; c < 4; c++)
{
Point2f p = Point2f(ImagePoints.at<float>(c,0), ImagePoints.at<float>(c,1));
imgPointsVector.push_back(p);
Point3f pObj = Point3f(ObjPoints.at<float>(c, 0), ObjPoints.at<float>(c, 1),ObjPoints.at<float>(c, 2));
objPointsVector.push_back(pObj);
}
</code></pre>
<p>and then update the call to solvePnP as</p>
<pre><code>cv::solvePnP(objPointsVector, imgPointsVector, camMatrix, distCoeff,raux,taux, false, CV_P3P);
</code></pre>
<p>I've just tested again, and P3P seems to be significantly less stable than CV_ITERATIVE as the marker is moved around. </p>
<p>Additionally, I generated an aruco board (i.e. many markers on a sheet of paper), and made the same change to the call to solvePnP in boardDetector.cpp. This time, since there are likely many available points, I was hoping to use EPNP, but the results from doing so are disappointing compared to CV_ITERATIVE. (In fact, they appear worse than just using a single maker with P3P.) </p>
<p>If you'd ...<span class="expander"> <a>(more)</a></span></p></div> http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?answer=260#post-id-260Hello, Alex!
I added these algorithms to the OpenCV. Also I am using them in my projects (but with Ransac scheme). If you can give me some data for solvePnP, that leads to incorrect result, I try to investigate it.Wed, 11 Jul 2012 13:36:04 -0500http://answers.opencv.org/question/258/is-anyone-using-p3p-or-epnp-in-solvepnp/?answer=260#post-id-260