I cannot figure out a problem with stereoRectify or initUndistortRectifyMap. It might be related to this question but the user didn't post enough information for me to determine it.
I do stereo calibration in two passes, because then we can split the intrinsic and extrinsic calibration, but it doesn't seem to work. These are inputs:
You can see radial lens distortion that needs to be corrected. So after each camera calibration I get a pretty good rectified images:
Left Camera Matrix: [1.0316489177476062e+003, 0., 6.5285947709947743e+002, 0., 1.0306889512875423e+003, 5.2195206055917618e+002, 0., 0., 1.]
Left Camera D: [-3.6474261084702730e-001, 1.7160954305339343e-001, 1.3998315284130826e-004, -5.7337307704477671e-005, -4.3398439779963091e-002]
Right Camera Matrix: [1.0294741711514341e+003, 0., 6.5282835377635274e+002, 0., 1.0285664286763802e+003, 5.2393079228919225e+002, 0., 0., 1..]
Right Camera D: [-3.8333204303394736e-001, 2.7030722876029428e-001, 1.2659691660140408e-004, -6.2717216712805863e-004, -1.4868158456944205e-001]
Now I do calibration together. I call stereoCalibrate like this:
double rms = stereoCalibrate(objectPoints, imagePoints[0], imagePoints[1],
cameraMatrix[0], distCoeffs[0],
cameraMatrix[1], distCoeffs[1],
Size(width,height), R, T, E, F,
cv::CALIB_USE_INTRINSIC_GUESS + cv::CALIB_FIX_INTRINSIC, TermCriteria(cv::TermCriteria::COUNT+cv::TermCriteria::EPS, 100, 1e-6));
RMS error: 0.716278 Average reprojection error: 0.932500
I get quite sane results. R is almost identity, showing that the cameras are very parallel:
9.9825297964692494e-001, -5.6272429338183161e-002, -1.8011172155504525e-002,
5.6492686748686502e-002, 9.9833134239913945e-001, 1.1962739128096707e-002,
1.7307945283909901e-002, -1.2959339485918969e-002, 9.9976621794804499e-001
And T is also showing correct results:
-2.8507174779791137e+002, -5.9310534218857187e+000, 9.6822612661982728e+000
It shows that the distance between cameras is 285mm, which is as precise as I can physically measure. So it seems all the inputs are correct. Now I call stereoRectify() like so:
stereoRectify(cameraMatrix[0], distCoeffs[0],
cameraMatrix[1], distCoeffs[1],
Size(width,height), R, T, R1, R2, P1, P2, Q,
cv::CALIB_ZERO_DISPARITY, 1.0, Size(width,height), &validRoi[0], &validRoi[1]);
R1:
9.9804912529642753e-001, -3.5033737187826153e-002, -5.1677662037772229e-002,
3.5384693276990765e-002, 9.9935639816774224e-001, 5.8917673825730115e-003,
5.1437991569745038e-002, -7.7088715031055979e-003, 9.9864643709544143e-001
R2:
9.9920772192907659e-001, 2.0788992329479110e-002, -3.3937386982939709e-002,
-2.1019454438531062e-002, 9.9975827258716765e-001, -6.4481725030493993e-003,
3.3795132377480969e-002, 7.1564091168283793e-003, 9.9940315930866519e-001
P1:
-4.7663940994070654e+003, 0., 4.8584126815795898e+003, 0.,
0., -4.7663940994070654e+003, 3.7089497299194336e+003, 0.,
0., 0., 1., 0.
P2:
-4.7663940994070654e+003, 0., 4.8584126815795898e+003, 1.3598416693461742e+006,
0., -4.7663940994070654e+003, 3.7089497299194336e+003, 0.,
0., 0., 1., 0.
Q:
1., 0., 0., -4.8584126815795898e+003,
0., 1., 0., -3.7089497299194336e+003,
0., 0., 0., -4.7663940994070654e+003,
0., 0., 3.5051096071344805e-003, 0.
Q also seems correct (1/B = 1/235 = 3.505e-003). But now when I use the remap function I get this:
I use OpenCV3.0. Maybe this was fixed in 3.1? I have calibrated stereo cameras with OpenCV before, and while it was painful it wasn't impossible. I have calibrated numerous times and I always get the same result. When I set Alpha to 0.0 in stereoRectify I get image like this:
But it is wrong as well. If we look at epipolar lines we can see they don't line up:
What am I doing wrong? It seems a lot of people cannot get stereoRectify to work. Any suggestions?