This forum is disabled, please visit https://forum.opencv.org

1 | initial version |

I tested your code with OpenCV 3.1. On my computer, I have two distinct problems:

- cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad result
- all PnP methods return bad result but if I turn off OPENCL (
`cv::setUseOptimized(false);`

), only cv::SOLVEPNP_ITERATIVE returns bad result

For the first issue, I guess that `cv::SOLVEPNP_ITERATIVE`

use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]
```

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, `cv::SOLVEPNP_ITERATIVE`

return completly wrong results.

For the second issue, if we use `cv::setUseOptimized(false);`

we get correct result for all methods except `cv::SOLVEPNP_ITERATIVE`

. For example with `cv::SOLVEPNP_DLS`

:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]
```

The strange thing on my computer is that if we use `cv::setUseOptimized(true);`

, and use `std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl;`

after the initialization of the value of the rotation, the behavior is the same as `cv::setUseOptimized(false);`

??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

- Model-Based Object Pose in 25 Lines of Code (The POSIT method)
- Complete Solution Classification for the Perspective-Three-Point Problem (
`SOLVEPNP_P3P`

) - EPnP: Efficient Perspective-n-Point Camera Pose Estimation (
`SOLVEPNP_EPNP`

) - A Direct Least-Squares (DLS) Method for PnP (
`SOLVEPNP_DLS`

) - Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (
`SOLVEPNP_UPNP`

) - and plenty of other methods that are not present in OpenCV 3.1.

Finally, could you try to disable OPENCL with `cv::setUseOptimized(false);`

as the first instruction and see if you get correct results with the other methods ?

2 | No.2 Revision |

Edit:

I just realize that I have copied / pasted your code and the 7th argument of `cv::solvePnP`

is commented. When I thought modify the solvePnP method, only the useExtrinsicGuess parameter was modified.

So, I have no issue with OpenCL and the results are the same regardless the value of `cv::setUseOptimized();`

on my computer.

With `cv::SOLVEPNP_DLS`

, `cv::SOLVEPNP_EPNP`

, `cv::SOLVEPNP_UPNP`

, the results are:

```
cv::useOptimized()=1
sum=44.8698093316393
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.02331595776374525, 0.0162874860835041, -0.002505115278821771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[1.3348540578548, 0.93479208297899, -0.132652269598486]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-37.49101751741978, -49.68669094111537, 809.825952217276]
```

The 3 methods return the same results because in the code it is in fact the EPnP method that is called.

There is a pull request to adress this issue: Reactivating SOLVEPNP_DLS and SOLVEPNP_UPNP.

With `cv::SOLVEPNP_P3P`

, the results are:

```
cv::useOptimized()=1
sum=0.205146641068635
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07112899814187482, 0.06897453111242667, -0.002581821704825981]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.07675884095018, 3.95387644149153, -0.00726862624869644]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32396572124871, -48.16931733672763, 769.072988680349]
```

With `cv::SOLVEPNP_ITERATIVE`

and `useExtrinsicGuess==false`

, the results are:

```
cv::useOptimized()=1
sum=8.9429127761996
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845, -0.01262643221321573, 3.115058790339473]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554, 48.15119764387534, -768.3810038581826]
```

With `cv::SOLVEPNP_ITERATIVE`

, `useExtrinsicGuess==true`

, `rotation2 = 0.0;`

and `translation2 = 0.0;`

, the results are:

```
cv::useOptimized()=1
sum=0
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390476; 0.06922079919505154; -0.002546761447480771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.02098269404469, 3.96792319109631, -0.00669089927595358]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332; -48.16989817918626; 769.0679944279347]
```

Maybe the unpredictable results on OSX is caused by the variables `rotation2`

and `translation2`

that were not explicitly initialized to zero, so when `useExtrinsicGuess==true`

, these variables contain random values from memory ?

I tested your code with OpenCV 3.1. ~~On my computer, I have two distinct problems:~~

- cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad
~~result~~result except`cv::SOLVEPNP_P3P`

~~all PnP methods return bad result but if I turn off OPENCL (~~`cv::setUseOptimized(false);`

), only cv::SOLVEPNP_ITERATIVE returns bad result

For the first issue, I guess that `cv::SOLVEPNP_ITERATIVE`

use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]
```

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, `cv::SOLVEPNP_ITERATIVE`

return completly wrong results.

~~For the second issue, if we use ~~`cv::setUseOptimized(false);`

we get correct result for all methods except `cv::SOLVEPNP_ITERATIVE`

. For example with `cv::SOLVEPNP_DLS`

:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]
```

~~The strange thing on my computer is that if we use ~~`cv::setUseOptimized(true);`

, and use `std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl;`

after the initialization of the value of the rotation, the behavior is the same as `cv::setUseOptimized(false);`

??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

- Model-Based Object Pose in 25 Lines of Code (The POSIT method)
- Complete Solution Classification for the Perspective-Three-Point Problem (
`SOLVEPNP_P3P`

) - EPnP: Efficient Perspective-n-Point Camera Pose Estimation (
`SOLVEPNP_EPNP`

) - A Direct Least-Squares (DLS) Method for PnP (
`SOLVEPNP_DLS`

) - Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (
`SOLVEPNP_UPNP`

) - and plenty of other methods that are not present in OpenCV 3.1.

~~Finally, could you try to disable OPENCL with ~~`cv::setUseOptimized(false);`

as the first instruction and see if you get correct results with the other methods ?

3 | No.3 Revision |

Edit:

I just realize that I have copied / pasted your code and the 7th argument of `cv::solvePnP`

is commented. When I thought modify the solvePnP method, only the useExtrinsicGuess parameter was modified.

So, I have no issue with OpenCL and the results are the same regardless the value of `cv::setUseOptimized();`

on my computer.

With `cv::SOLVEPNP_DLS`

, `cv::SOLVEPNP_EPNP`

, `cv::SOLVEPNP_UPNP`

, the results are:

```
cv::useOptimized()=1
sum=44.8698093316393
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.02331595776374525, 0.0162874860835041, -0.002505115278821771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[1.3348540578548, 0.93479208297899, -0.132652269598486]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-37.49101751741978, -49.68669094111537, 809.825952217276]
```

The 3 methods return the same results because in the code it is in fact the EPnP method that is called.

There is a pull request to adress this issue: Reactivating SOLVEPNP_DLS and SOLVEPNP_UPNP.

With `cv::SOLVEPNP_P3P`

, the results are:

```
cv::useOptimized()=1
sum=0.205146641068635
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07112899814187482, 0.06897453111242667, -0.002581821704825981]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.07675884095018, 3.95387644149153, -0.00726862624869644]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32396572124871, -48.16931733672763, 769.072988680349]
```

With `cv::SOLVEPNP_ITERATIVE`

and `useExtrinsicGuess==false`

, the results are:

```
cv::useOptimized()=1
sum=8.9429127761996
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845, -0.01262643221321573, 3.115058790339473]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554, 48.15119764387534, -768.3810038581826]
```

With `cv::SOLVEPNP_ITERATIVE`

, `useExtrinsicGuess==true`

, `rotation2 = 0.0;`

and `translation2 = 0.0;`

, the results ~~are:~~are (only the method `cv::SOLVEPNP_ITERATIVE`

accepts an initial guess for the pose):

```
cv::useOptimized()=1
sum=0
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390476; 0.06922079919505154; -0.002546761447480771]
True_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
Est_euler=[4.02098269404469, 3.96792319109631, -0.00669089927595358]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332; -48.16989817918626; 769.0679944279347]
```

~~Maybe ~~If in your code the parameter `useExtrinsicGuess`

is also commented, maybe the unpredictable results on OSX is caused by the variables `rotation2`

and `translation2`

that were not explicitly initialized to zero, so when `useExtrinsicGuess==true`

, these variables contain random values from memory ?

I tested your code with OpenCV 3.1. ~~On my computer, I have two distinct problems:~~

- cv::SOLVEPNP_ITERATIVE returns bad result, all PnP methods return bad result except
`cv::SOLVEPNP_P3P`

~~all PnP methods return bad result but if I turn off OPENCL (~~`cv::setUseOptimized(false);`

), only cv::SOLVEPNP_ITERATIVE returns bad result

For the first issue, I guess that `cv::SOLVEPNP_ITERATIVE`

use an initial pose before iteratively minimizing the reprojection error.

In the code, we can see that cv::SOLVEPNP_ITERATIVE use cvFindExtrinsicCameraParams2 which use Direct Linear Transform (DLT) when the points are non coplanar to estimate an initial pose before iteratively refines the pose using the Levenberg-Marquardt method.

My guess is that the DLT returns a bad initial pose (due to a singular configuration ?) and then we diverge in the minimization process. The odd thing is that if we look at the rotation and translation vector estimated we have:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[-0.3014129665370845; -0.01262643221321573; 3.115058790339473]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[-0.536089671153311, 11.0500652280158, 179.259679528549]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[34.50784095206554; 48.15119764387534; -768.3810038581826]
```

The estimated pose seems behind the camera if we look at the translation vector if I am not wrong ? Maybe it is due to the initial pose computed by the DLT method ?

Also, if we don't cast to float values, `cv::SOLVEPNP_ITERATIVE`

return completly wrong results.

~~For the second issue, if we use ~~`cv::setUseOptimized(false);`

we get correct result for all methods except `cv::SOLVEPNP_ITERATIVE`

. For example with `cv::SOLVEPNP_DLS`

:

```
True rotation=[0.07015543431043625, 0.06922079622745514, -0.002546760952100158]
Est rotation=[0.07015537539390418, 0.06922079919505153, -0.002546761447480788]
true_euler=[4.02098607576823, 3.96792301903936, -0.00669075977245175]
est_euler=[4.02098269404469, 3.9679231910964, -0.00669089933042529]
True translation=[-35.32360076904297, -48.16989898681641, 769.0679931640625]
Est translation=[-35.32360147801332, -48.16989817918626, 769.0679944279347]
```

~~The strange thing on my computer is that if we use ~~`cv::setUseOptimized(true);`

, and use `std::cout << "norm(rotation)=" << cv::norm(rotation) << std::endl;`

after the initialization of the value of the rotation, the behavior is the same as `cv::setUseOptimized(false);`

??? This leads me to suspect a possible issue with OPENCL.

From my knowledge, the pose estimation problem (or PnP problem) is a non linear problem. Thus, it is a non trivial problem and it is always possible to converge to a local minima I think. That's why there are plenty of methods to estimate the camera pose:

- Model-Based Object Pose in 25 Lines of Code (The POSIT method)
- Complete Solution Classification for the Perspective-Three-Point Problem (
`SOLVEPNP_P3P`

) - EPnP: Efficient Perspective-n-Point Camera Pose Estimation (
`SOLVEPNP_EPNP`

) - A Direct Least-Squares (DLS) Method for PnP (
`SOLVEPNP_DLS`

) - Exhaustive Linearization for Robust Camera Pose and Focal Length Estimation (
`SOLVEPNP_UPNP`

) - and plenty of other methods that are not present in OpenCV 3.1.

~~Finally, could you try to disable OPENCL with ~~`cv::setUseOptimized(false);`

as the first instruction and see if you get correct results with the other methods ?

Copyright OpenCV foundation, 2012-2018. Content on this site is licensed under a Creative Commons Attribution Share Alike 3.0 license.