I'd like to know the accuracy of the camera's calibration status in millimeter units. In other words, what does the X pixels error (the Root Mean Square error) in the 2D image correspond to the distance in millimeters in the 3D object coordinate system?
In OpenCV, I can get the accuracy of the calibration based on the Root Mean Square error (pixel units) as a result of calibrateCamera function. Or, I can manually calculate by reprojecting the object points to the image with projectPoints and by comparing them with the current image points.
For example, let's say I got the RMS error around 2.0 (I mean I already have 2d projection error in pixel units). Does this mean 5mm or 10mm difference? How to calculate the conversion from pixels in 2d to millimeters in 3d and to get mm units error in 3D space? Note that the ArUco marker is always placed horizontally and I know the size, position, and pose of the marker. Could you let me know how to calculate the error in x and y axes in the object frame? Please let me know if this question doesn't make sense or if you need further information.