I actually managed to solve this problem. The general idea was correct, but:
- I was not understanding correctly the vector rotation notation the robot was giving. It was necessary to multiply the actual values by a factor.
- I created a new program that extracts directly from the robot and the pictures the matrixes that the algorithm requires and writes these values to a YML file.
- The CALIB_HAND_EYE_TSAI method wasn't giving me correct values. But with the four others, the values seem to converge to the actual values
Anyway, thank you for your help. I am stuck to get more precision in the algorithm, but that's for another question.