Overhead camera's pose estimation with OpenCV SolvePnP results in a few centimeter off height
I'd like to get the pose (translation: x, y, z and rotation: Rx, Ry, Rz in World coordinate system) of the overhead camera. I got many object points and image points by moving the ChArUco calibration board with a robotic arm (like this https://www.youtube.com/watch?v=8q99d...). Because of that, I already have exact positions of all the object points.
In order to feed many points to solvePnP, I set the first detected pattern (ChArUco board) as the first object and used it as the object coordinate system's origin. Then, I added the detected object points (from the second pattern to the last) to the first detected object points' coordinate system (the origin of the object frame is the origin of the first object).
After I got the transformation between the camera and the object's coordinate frame, I calculated the camera's pose based on that transformation.
The result looked pretty good at first, but when I measured the camera's absolute pose by using a ruler or a tape measure, I noticed that the extrinsic calibration result was around 15-20 millimeter off for z direction (the height of the camera), though almost correct for the others (x, y, Rx, Ry, Rz). The result was same even I changed the range of the object points by moving a robotic arm differently, it always ended up to have a few centimeters off for the height.
Has anyone experienced the similar problem before? I'd like to know anything I can try. What is the common mistake when the depth direction (z) is inaccurate?
solvePnPreturns the rotation and translation that allows to transform a point expressed in the object frame into the camera frame. The camera frame should somewhere inside of the optical lens. Did you measure at this location?Also, the pose estimation depends on the accuracy of the intrinsic parameters and the distance to the object / size of the object in the image.
Thank you for your comment Eduardo. I measured the front face of the lens as the origin of the camera frame. The calibration result was around 15-20 millimeter small. If the result was 15-20 millimeter big, I could think that the origin is in the camera, but seems the result said the origin was 15-20 mm in front of the camera.
Regarding the intrinsic parameters, I'm using the predefined camera matrix and distortion parameters, which are set in the factory. Do you think this is the case?
The distance to the object was around 800 millimeter.
Remember, OpenCV uses a pinhole camera model. This is not actually a representation of how the lenses actually work. So it's entirely possible that the actual lens is longer or shorter than the focal length. You would need to measure that difference and include it in your calculations.
See the wikipedia page for telephoto lens for examples.
I was using a ZED camera from SteleoLabs. Focal length was around 1,400mm and it's longer than the distance to the marker. So doesn't this use a pinhole camera model?
Umm, 1400 mm is an incredibly long focal length. That's like those pods you see under helicopters for watching things miles away. Do you mean the camera matrix has 1400 in it? That would be 1400 pixels, and then multiplied by the size of the pixel on the FPA, which is usually 5-50 micrometer range.
Sorry, you are right. It was 1400 pixel.
So my suggestion is to check at several different heights, and see if there is a constant offset between measured and calculated. If there is, then that's the difference between the physical length and the focal length. Then you just apply that offset and you're done.
If it's not, show us a plot and maybe we can figure out what's up.
After spending a lot of time, I noticed that the output from the camera was rectified by default. Using the unrectified image solved this issue, though I still have a centimeter off for y-direction in the camera frame when I get pointcloud data from the camera.