Why both stereoRectify-workflows give different result?
I get different result in both workflows, which should be equal (in my opinion).
- First workflow: First i remove the distortion in the images, and then in the following functions in the workflow i use zero-distortion (e.g. Mat() as parameter).
- Second workflow: I don't remove the distortion, but instead of it I use the distortion coefficients in the following functions (in stereoRectify() and in initUndistortRectifyMap()).
First workflow (with inital undistortion):
undistort(image1, image1, camera_matrix1, distCoeffs1);
undistort(image2, image2, camera_matrix2, distCoeffs2);
....
E = findEssentialMat(image_points1, image_points2, camera_matrix1, RANSAC);
recoverPose(E, image_points1, image_points2, camera_matrix1, R, T);
stereoRectify(camera_matrix1, Mat(), camera_matrix2, Mat(), image1.size(), R, T, R1, R2, Proj1, Proj2, Q);
Mat mapx1, mapy1;
initUndistortRectifyMap(camera_matrix1, Mat(), R1, Proj1, image1.size(), CV_16SC2, mapx1, mapy1);
remap(image1, image1_rectified, mapx1, mapy1, INTER_LINEAR);
Mat mapx2, mapy2;
initUndistortRectifyMap(camera_matrix2, Mat(), R2, Proj2, image2.size(), CV_16SC2, mapx2, mapy2);
remap(image2, image2_rectified, mapx2, mapy2, INTER_LINEAR);
Resulting disparity map:
Second workflow (without inital undistiortion):
E = findEssentialMat(image_points1, image_points2, camera_matrix1, RANSAC);
recoverPose(E, image_points1, image_points2, camera_matrix1, R, T);
stereoRectify(camera_matrix1, distCoeffs1, camera_matrix2, distCoeffs2, image1.size(), R, T, R1, R2, Proj1, Proj2, Q);
Mat mapx1, mapy1;
initUndistortRectifyMap(camera_matrix1, distCoeffs1, R1, Proj1, image1.size(), CV_16SC2, mapx1, mapy1);
remap(image1, image1_rectified, mapx1, mapy1, INTER_LINEAR);
Mat mapx2, mapy2;
initUndistortRectifyMap(camera_matrix2, distCoeffs2, R2, Proj2, image2.size(), CV_16SC2, mapx2, mapy2);
remap(image2, image2_rectified, mapx2, mapy2, INTER_LINEAR);
Resulting disparity map:
Then i use these rectified images to calculate the disparity map, but i get different results in both workflows.
The second workflow looks like it gives better results (at least on the wall on the right side).
I would expect that both workflows do get the same results...
Can you give E and F in both case? or use same value (with RANSAC you have to check convergence) Are you sure that results shown are not normalized
The only difference in both workflows is, that in the first one i undistort the images, and then in all following steps use zero distortion coefficients, and in the second workflow i don't undistort the images, but use the distortion coefficients in following steps.
First workflow:
Second workflow:
if you add two images there is a small shift. Can you compare left and right image in both case instead of disparity map?
They don't seem be perfectly aligned in both cases...
Does the
undistort(image1, image1, camera_matrix1, distCoeffs1);
function just removes the lens distortion or does it also changes the cx, cy values (shifting the images to the principal point) ?I think you found "Does the undistort(image1, image1, camera_matrix1, distCoeffs1);"
undistort(image1, image1, camera_matrix1, distCoeffs1); it means scale is changing there is a perspective transformation. Now when you give an empty distorsion mat you give too a perspective transform and scale is changed (I think fx/fy is not equal to 1 but not very far).