Ask Your Question
1

Different output from Python and C++ in CalibrateHandEye()

asked 2020-10-14 09:56:38 -0500

rgade gravatar image

Hi,

We have tried to translate a C++ project into Python. The problem is that we get different outputs from the calibrateHandEye() function in the two versions. Even though the input is the same of course.

The code of interest is this part (C++ version)

Mat rotationCamera2Gripper, translationCamera2Gripper;

calibrateHandEye(rotationGripper2Base, translationGripper2Base, rvecsRod, tvecs,
                 rotationCamera2Gripper, translationCamera2Gripper, CALIB_HAND_EYE_TSAI);

Mat rotVecCam2Gripper;
Rodrigues(rotationCamera2Gripper, rotVecCam2Gripper);

std::cout << "Rot: " << std::endl << rotVecCam2Gripper << std::endl;
std::cout << "Trans: " << std::endl << translationCamera2Gripper << std::endl;

and in Python:

rotationCamera2Gripper, translationCamera2Gripper = cv.calibrateHandEye(
                    rotationGripper2Base, 
                    translationGripper2Base,
                    rvecsRod, 
                    tvecs, 
                    method=cv.CALIB_HAND_EYE_TSAI)


rotVecCam2Gripper, _ = cv.Rodrigues(rotationCamera2Gripper)

print("Rot:")
print(rotVecCam2Gripper)
print("Trans: ")
print(translationCamera2Gripper)

The main difference is between the translation vectors. Output from C++:

image description

Output from Python:

image description

We end up in a situation where everything before calibrateHandEye seem to be the same, but the output is different. Any hint to what could be the problem is appreciated :)

edit retag flag offensive close merge delete

Comments

How did you install Python OpenCV? If you are using pip install, you are not using the same config (e.g. Math libraries, version, etc.) between C++ OpenCV that you may have built from source and https://github.com/skvark/opencv-python.

Eduardo gravatar imageEduardo ( 2020-10-15 04:50:52 -0500 )edit

the rotation vectors look identical but the translation vectors differ significantly. speculation: your the code in both language versions is not equivalent (check intermediate results!); you are actually not giving it the same input even though you think you do; you are feeding it too little data, and ill conditioned data for the algorithm; the algorithm contains some randomization (pose estimation) and that causes random behavior in either version when run repeatedly; the source in OpenCV has bugs that cause diverging behavior for minor disturbances (always a possibility in functions that aren't commonly used)

crackwitz gravatar imagecrackwitz ( 2020-10-15 06:27:36 -0500 )edit

We used pip install for Python, and standard binaries for C++ (Visual Studio). We checked that we used same version of OpenCV (4.4.0) for both languages.

We import exactly the same data for both programs imported with FileStorage, and all other steps produce the same results. We use 16 poses for the calibration, and the results are consistent when run repeatedly (and on different computers/software versions). But there may be some problem with the data being ill conditioned...

The only thing we can't check is the OpenCV code that is called from the Python version of calibrateHandEye, as we haven't found a way to debug from Python into the original C++ functions.

rgade gravatar imagergade ( 2020-10-15 06:38:32 -0500 )edit

What I would do:

  • remove Python OpenCV installed by pip
  • build OpenCV with Python bindings, can be done easily even on Windows
  • you should get now identical results between C++ and Python interfaces
  • if not, there is an issue with the Python bindings or in your code
Eduardo gravatar imageEduardo ( 2020-10-15 08:19:41 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-10-18 05:11:30 -0500

rgade gravatar image

We found the problem. It was the datatype of the input translation vector. In C++ it was a Mat array of size [1,3], but in Python it needed to be a NumPy array of size [3].

edit flag offensive delete link more

Comments

Could you please publish the rest of your Python Code? I am also trying to make a Handeye calibration but could not find that many Python examples.

luenmb gravatar imageluenmb ( 2020-10-23 04:45:29 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-10-14 09:56:38 -0500

Seen: 52 times

Last updated: Oct 18