With OpenGL, I created a virtual world with objects and a camera. I made a list of tuples (an object's world_coord x y z, an object's pixel_coord x y) where z = 0.
I calculated rvec and tvec with solvePnP/Ransac expecting zero reprojection error (+ maybe some floating error) but the reprojection result is terrible, even with >=10 non-colinear sample points. This bug seems related: https://github.com/opencv/opencv/issues/6117
My question is: once the bug is fixed, can I expect a zero reprojection error? Or is the error inherent because the algorithms are incremental and approximated? Isn't there an algorithm that guarantees the global minimization of some error metric (e.g. L2)?
It is surprising that all PnP algorithms fail. Unlike solvePnP, findHomography() worked really well even with some noisy data samples.