OpenCV projectionPoints giving abnormal results

asked 2019-09-11 01:35:53 -0500

manishg gravatar image

I've did Camera calibration using fisheye model. I did both intrinsic and extrinsic calibration using checkerboard images and got the values K (Camera matrix), D(dist coeffs), R and T. Reprojection error for intrinsic calibration is ~2.45 and for extrinsic calibration it's ~33.62.

Now the task i'm trying to do is to obtain the uv mapping for the 3D points on a particular surface so that i can project the image on to the surface using openGL. To get the 2D uv point corresponding to the 3D point i'm using opencv fisheye models projectPoints() method projectPoints

Image i used for extrinsinsic calibration is of size 3024 * 3024 pixels

Functions i used for intrinsic and extrinsic calibration:

fisheye::calibrate method to get K and D:

double rms = fisheye::calibrate(objectPoints, imagePoints, imageSize, cameraMatrix,
                    distCoeffs, rvecs, tvecs, flags|cv::CALIB_FIX_K4|cv::CALIB_FIX_K5|

solvePnP method to get R and T:

bool ok = solvePnP(objectPoints,imagePoints,cameraMatrix,distCoeffs,rvecs,tvecs,false,flags);

I'm getting correct uv points corresponding to the 3D points which are projected on to the image. But when the uv points goes out of the image borders it's returning very deviated and uncorrelatable values like below:

projectPoints function call:

fisheye::projectPoints(mesh_points, uv_points, rvecs, tvecs,
                cameraMatrix, distCoeffs); //here mesh_points are 3D points corresponding to the surface

3D points i've passed to project points function:

1) 0.0 0.0 0.0
2) -11.2 0.0 0.0
3) -11.3 0.0 0.0

UV points i got in the result:

1)  1962.62 1030.9
2)  4734.26 390.099
3) -1781.13 2592.06

So for points (-11.2,0,0) and (-11.3,0,0) the difference between the uv points returned by projectPoints is huge. But ideally as these points are very close in 3D space their uv points should also be close to each other. I'm not able to understand this response from the projectPoints method. The uv point returned for (0,0,0) and (-11.2,0,0) are correct when i compared manually on the image but the uv point corresponding to (-11.3,0,0) is very far and incosistent from other uv points. I feel at the end it's a matrix multiplication that should be happened in project points method and that should be consisent.

Please help us if you've any information regarding this problem.

edit retag flag offensive close merge delete


The intrinsic reprojection error is high, and the error from SolvePNP is absurdly high. I'd start by trying to address that (particularly the extrinsics), but if I had to guess, you didn't get enough calibration points near the edges/corners of the image, and the resulting distortion model works well for the central part of the image but not for points far from the center.

I haven't used the fisheye model much myself, but with the standard distortion models you can get situations where the model is well behaved for much of the image, but then becomes unstable around a certain radius. (results from cv::undistort will produce images with a circular artifact with very high distortion/smearing). I suggest using a CHARUCO calibration target so you can use points all the way to the corners

swebb_denver gravatar imageswebb_denver ( 2019-09-11 14:39:53 -0500 )edit