Revision history - OpenCV Q&A Forum

I tested your code with OpenCV 3.1. On my computer, I have two distinct problems:

cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad result
all PnP methods return bad result but if I turn off OPENCL (cv::setUseOptimized(false);), only cv::SOLVEPNP_ITERATIVE returns bad result

For the first issue, I guess that cv::SOLVEPNP_ITERATIVE use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, cv::SOLVEPNP_ITERATIVE return completly wrong results.

For the second issue, if we use cv::setUseOptimized(false); we get correct result for all methods except cv::SOLVEPNP_ITERATIVE. For example with cv::SOLVEPNP_DLS:

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]

The strange thing on my computer is that if we use cv::setUseOptimized(true);, and use std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl; after the initialization of the value of the rotation, the behavior is the same as cv::setUseOptimized(false); ??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

Model-Based Object Pose in 25 Lines of Code (The POSIT method)
Complete Solution Classification for the Perspective-Three-Point Problem (SOLVEPNP_P3P)
EPnP: Efficient Perspective-n-Point Camera Pose Estimation (SOLVEPNP_EPNP)
A Direct Least-Squares (DLS) Method for PnP (SOLVEPNP_DLS)
Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (SOLVEPNP_UPNP)
and plenty of other methods that are not present in OpenCV 3.1.

Finally, could you try to disable OPENCL with cv::setUseOptimized(false); as the first instruction and see if you get correct results with the other methods ?

Edit:

I just realize that I have copied / pasted your code and the 7th argument of cv::solvePnP is commented. When I thought modify the solvePnP method, only the useExtrinsicGuess parameter was modified.

So, I have no issue with OpenCL and the results are the same regardless the value of cv::setUseOptimized(); on my computer.

With cv::SOLVEPNP_DLS, cv::SOLVEPNP_EPNP, cv::SOLVEPNP_UPNP, the results are:

cv::useOptimized()=1
sum=44.8698093316393
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.02331595776374525, 0.0162874860835041, -0.002505115278821771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[1.3348540578548, 0.93479208297899, -0.132652269598486]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-37.49101751741978, -49.68669094111537, 809.825952217276]

The 3 methods return the same results because in the code it is in fact the EPnP method that is called.

There is a pull request to adress this issue: Reactivating SOLVEPNP_DLS and SOLVEPNP_UPNP.

With cv::SOLVEPNP_P3P, the results are:

cv::useOptimized()=1
sum=0.205146641068635
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07112899814187482, 0.06897453111242667, -0.002581821704825981]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.07675884095018, 3.95387644149153, -0.00726862624869644]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32396572124871, -48.16931733672763, 769.072988680349]

With cv::SOLVEPNP_ITERATIVE and useExtrinsicGuess==false, the results are:

cv::useOptimized()=1
sum=8.9429127761996
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845, -0.01262643221321573, 3.115058790339473]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554, 48.15119764387534, -768.3810038581826]

With cv::SOLVEPNP_ITERATIVE, useExtrinsicGuess==true, rotation2 = 0.0; and translation2 = 0.0;, the results are:

cv::useOptimized()=1
sum=0
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390476; 0.06922079919505154; -0.002546761447480771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.02098269404469, 3.96792319109631, -0.00669089927595358]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332; -48.16989817918626; 769.0679944279347]

Maybe the unpredictable results on OSX is caused by the variables rotation2 and translation2 that were not explicitly initialized to zero, so when useExtrinsicGuess==true, these variables contain random values from memory ?

I tested your code with OpenCV 3.1. On my computer, I have two distinct problems:

cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad ~~result~~result except cv::SOLVEPNP_P3P
all PnP methods return bad result but if I turn off OPENCL (cv::setUseOptimized(false);), only cv::SOLVEPNP_ITERATIVE returns bad result

For the first issue, I guess that cv::SOLVEPNP_ITERATIVE use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, cv::SOLVEPNP_ITERATIVE return completly wrong results.

For the second issue, if we use cv::setUseOptimized(false); we get correct result for all methods except cv::SOLVEPNP_ITERATIVE. For example with cv::SOLVEPNP_DLS:

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]

The strange thing on my computer is that if we use cv::setUseOptimized(true);, and use std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl; after the initialization of the value of the rotation, the behavior is the same as cv::setUseOptimized(false); ??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

Model-Based Object Pose in 25 Lines of Code (The POSIT method)
Complete Solution Classification for the Perspective-Three-Point Problem (SOLVEPNP_P3P)
EPnP: Efficient Perspective-n-Point Camera Pose Estimation (SOLVEPNP_EPNP)
A Direct Least-Squares (DLS) Method for PnP (SOLVEPNP_DLS)
Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (SOLVEPNP_UPNP)
and plenty of other methods that are not present in OpenCV 3.1.

Finally, could you try to disable OPENCL with cv::setUseOptimized(false); as the first instruction and see if you get correct results with the other methods ?

Edit:

I just realize that I have copied / pasted your code and the 7th argument of cv::solvePnP is commented. When I thought modify the solvePnP method, only the useExtrinsicGuess parameter was modified.

So, I have no issue with OpenCL and the results are the same regardless the value of cv::setUseOptimized(); on my computer.

With cv::SOLVEPNP_DLS, cv::SOLVEPNP_EPNP, cv::SOLVEPNP_UPNP, the results are:

cv::useOptimized()=1
sum=44.8698093316393
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.02331595776374525, 0.0162874860835041, -0.002505115278821771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[1.3348540578548, 0.93479208297899, -0.132652269598486]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-37.49101751741978, -49.68669094111537, 809.825952217276]

The 3 methods return the same results because in the code it is in fact the EPnP method that is called.

There is a pull request to adress this issue: Reactivating SOLVEPNP_DLS and SOLVEPNP_UPNP.

With cv::SOLVEPNP_P3P, the results are:

cv::useOptimized()=1
sum=0.205146641068635
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07112899814187482, 0.06897453111242667, -0.002581821704825981]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.07675884095018, 3.95387644149153, -0.00726862624869644]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32396572124871, -48.16931733672763, 769.072988680349]

With cv::SOLVEPNP_ITERATIVE and useExtrinsicGuess==false, the results are:

cv::useOptimized()=1
sum=8.9429127761996
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845, -0.01262643221321573, 3.115058790339473]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554, 48.15119764387534, -768.3810038581826]

With cv::SOLVEPNP_ITERATIVE, useExtrinsicGuess==true, rotation2 = 0.0; and translation2 = 0.0;, the results ~~are:~~are (only the method cv::SOLVEPNP_ITERATIVE accepts an initial guess for the pose):

cv::useOptimized()=1
sum=0
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390476; 0.06922079919505154; -0.002546761447480771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.02098269404469, 3.96792319109631, -0.00669089927595358]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332; -48.16989817918626; 769.0679944279347]

~~Maybe~~ If in your code the parameter useExtrinsicGuess is also commented, maybe the unpredictable results on OSX is caused by the variables rotation2 and translation2 that were not explicitly initialized to zero, so when useExtrinsicGuess==true, these variables contain random values from memory ?

I tested your code with OpenCV 3.1. ~~On my computer, I have two distinct problems:~~

cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad result except cv::SOLVEPNP_P3P
~~all PnP methods return bad result but if I turn off OPENCL (cv::setUseOptimized(false);), only cv::SOLVEPNP_ITERATIVE returns bad result~~

For the first issue, I guess that cv::SOLVEPNP_ITERATIVE use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, cv::SOLVEPNP_ITERATIVE return completly wrong results.

~~For the second issue, if we use cv::setUseOptimized(false); we get correct result for all methods except cv::SOLVEPNP_ITERATIVE. For example with cv::SOLVEPNP_DLS:~~

True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]

The strange thing on my computer is that if we use cv::setUseOptimized(true);, and use std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl; after the initialization of the value of the rotation, the behavior is the same as cv::setUseOptimized(false); ??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

Model-Based Object Pose in 25 Lines of Code (The POSIT method)
Complete Solution Classification for the Perspective-Three-Point Problem (SOLVEPNP_P3P)
EPnP: Efficient Perspective-n-Point Camera Pose Estimation (SOLVEPNP_EPNP)
A Direct Least-Squares (DLS) Method for PnP (SOLVEPNP_DLS)
Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (SOLVEPNP_UPNP)
and plenty of other methods that are not present in OpenCV 3.1.

~~Finally, could you try to disable OPENCL with cv::setUseOptimized(false); as the first instruction and see if you get correct results with the other methods ?~~

Revision history [back]