Why both stereoRectify-workflows give different result?

asked 2017-05-02 07:16:38 -0500

mirnyy gravatar image

updated 2017-05-02 07:52:49 -0500

I get different result in both workflows, which should be equal (in my opinion).

  1. First workflow: First i remove the distortion in the images, and then in the following functions in the workflow i use zero-distortion (e.g. Mat() as parameter).
  2. Second workflow: I don't remove the distortion, but instead of it I use the distortion coefficients in the following functions (in stereoRectify() and in initUndistortRectifyMap()).

First workflow (with inital undistortion):

undistort(image1, image1, camera_matrix1, distCoeffs1);
undistort(image2, image2, camera_matrix2, distCoeffs2);
....
E = findEssentialMat(image_points1, image_points2, camera_matrix1, RANSAC);
recoverPose(E, image_points1, image_points2, camera_matrix1, R, T);

stereoRectify(camera_matrix1, Mat(), camera_matrix2, Mat(), image1.size(), R, T, R1, R2, Proj1, Proj2, Q);

Mat mapx1, mapy1;
initUndistortRectifyMap(camera_matrix1, Mat(), R1, Proj1, image1.size(), CV_16SC2, mapx1, mapy1);
remap(image1, image1_rectified, mapx1, mapy1, INTER_LINEAR);

Mat mapx2, mapy2;
initUndistortRectifyMap(camera_matrix2, Mat(), R2, Proj2, image2.size(), CV_16SC2, mapx2, mapy2);
remap(image2, image2_rectified, mapx2, mapy2, INTER_LINEAR);

Resulting disparity map: image description

Second workflow (without inital undistiortion):

E = findEssentialMat(image_points1, image_points2, camera_matrix1, RANSAC);
recoverPose(E, image_points1, image_points2, camera_matrix1, R, T);

stereoRectify(camera_matrix1, distCoeffs1, camera_matrix2, distCoeffs2, image1.size(), R, T, R1, R2, Proj1, Proj2, Q);

Mat mapx1, mapy1;
initUndistortRectifyMap(camera_matrix1, distCoeffs1, R1, Proj1, image1.size(), CV_16SC2, mapx1, mapy1);
remap(image1, image1_rectified, mapx1, mapy1, INTER_LINEAR);

Mat mapx2, mapy2;
initUndistortRectifyMap(camera_matrix2, distCoeffs2, R2, Proj2, image2.size(), CV_16SC2, mapx2, mapy2);
remap(image2, image2_rectified, mapx2, mapy2, INTER_LINEAR);

Resulting disparity map: image description

Then i use these rectified images to calculate the disparity map, but i get different results in both workflows.
The second workflow looks like it gives better results (at least on the wall on the right side).
I would expect that both workflows do get the same results...

edit retag flag offensive close merge delete

Comments

Can you give E and F in both case? or use same value (with RANSAC you have to check convergence) Are you sure that results shown are not normalized

LBerger gravatar imageLBerger ( 2017-05-02 07:31:52 -0500 )edit

The only difference in both workflows is, that in the first one i undistort the images, and then in all following steps use zero distortion coefficients, and in the second workflow i don't undistort the images, but use the distortion coefficients in following steps.

First workflow:

E:
[0.001475644405888272, 0.1181647875451908, -0.0112713504543981;
 -0.03422232553524984, -0.01563004061572099, -0.7060516543669478;
 0.01120162853491419, 0.6968840502993794, -0.0145121226712516]

Second workflow:

E:
[0.001124820410028303, 0.1233849494645024, -0.009042198431885057;
 -0.0451438842573884, -0.01493352749849507, -0.7054767929444381;
 0.008692357842647526, 0.6960305595704939, -0.0141613847756494]
mirnyy gravatar imagemirnyy ( 2017-05-02 07:44:49 -0500 )edit

if you add two images there is a small shift. Can you compare left and right image in both case instead of disparity map?

LBerger gravatar imageLBerger ( 2017-05-02 08:13:09 -0500 )edit
1

They don't seem be perfectly aligned in both cases...
Does the undistort(image1, image1, camera_matrix1, distCoeffs1); function just removes the lens distortion or does it also changes the cx, cy values (shifting the images to the principal point) ?

mirnyy gravatar imagemirnyy ( 2017-05-02 09:04:29 -0500 )edit

I think you found "Does the undistort(image1, image1, camera_matrix1, distCoeffs1);"

undistort(image1, image1, camera_matrix1, distCoeffs1); it means scale is changing there is a perspective transformation. Now when you give an empty distorsion mat you give too a perspective transform and scale is changed (I think fx/fy is not equal to 1 but not very far).

LBerger gravatar imageLBerger ( 2017-05-02 09:58:51 -0500 )edit