Revision history [back]

I have a 2 camera stereo setup where i calibrated each camera using opencv's initCameraMatrix2D then stereocalibrate to get the camera matrices K1, K2 and fundamental matrix F.
I tested it using a checker board and the correspondence equation where X't.F.X = 0 and i was able to get 0.00# so i believe this means the intrinsic calibration is correct?

Next i fixed an ArUco marker on the desired world origin and used the estimatePoseSingleMarker to get the RVec and TVec for both cameras and used them to get the projection Matrices P1 and P2 where P = k[R|T] and R is Rodrigues(RVec,R) and T is Tvec directly. This should make my marker center be my world origin?

Now i want to do triangulation so i used the triangulatePoints methond.

I detected the 4 marker corners from both cameras and triangulated them. My ArUco makrer size is 9 cm so each corner coordinate should be located at (+-0.045 , +- 0.045, 0) since the marker center is supposed to be my world origin but as a result of triangulation, i get:
(-0.0482, 0.0678, 0.0028)
( 0.0420, 0.0659, 0.0061)
( 0.0414,-0.0167, 0.0059)
(-0.0476,-0.0167, 0.0047)
This is of course after converting points from homogeneous coordinates using convertPointsFromHomogeneous. The Y values seems way off by a few cm.

I created 5 points with location at (0,0,0,1) and the (+-0.045 , +- 0.045, 0, 1) and projected them by multiplying by projection matrices and i draw the projected pixel coordinates but they are shifted in the image a bit from the expected location

and

To test the triangulation:

• I detected the 4 marker corners from both cameras and triangulated them then reprojected the traingulated points back to the image plane using projectedPoints = P * traingulatedPoints and as a result i get back almost exactly the same original marker pixel coordinates that i detected before triangulation and reprojection. This should mean that my calibration, Projection Matrices and triangulation are correct. (or not?)

and (red is the detected corners and blue is the projection of the triangulated corners).

So i dont get what is wrong. Maybe the trinagulation result is correct but in a different coordinate system not using the marker origin as my center? if so, how can i get the triangulated points results relative to my marker's origin? does it have to do with the need to undistort the points? Is using the output Rvec and Tvec from the aruco marker good way to get a correct extrinsic matrix?