System information (version)
- OpenCV => 3.4.1
- Operating System / Platform => Windows 64 Bit
- Compiler => Visual Studio 2013 + Python Bindings
Detailed description
When using stereoRectify, the resulting projection matrices depend on the input distortion, which in my understanding should not happen.
I encountered the problem when using stereoRectify on a new pair of cameras. I have used the whole pipeline successfully before on cameras of the same type. This time, the results returned by stereRectify are wrong. The camera centers in the new projection matrices have huge values (see values below).
To locate the problem, I run the rectification and undistortion in two individual steps, instead of using stereRectify. If I first undistort the images and then feed the new undistorted images and a vector filled with zeros as distortion into stereRectify I obtain good results. See the Steps to reproduce.
Steps to reproduce
My data input:
M_L = [[ 430. 0. 200.23631705]
[ 0. 430. 199.36536477]
[ 0. 0. 1. ]]
M_L.dtype = float64
dist_L = [-0.52501353 0.50232865 0.01204062 -0.03013492 -0.42135838]
dist_L.dtype = float64
M_R = [[ 430. 0. 199.64316951]
[ 0. 430. 199.40004749]
[ 0. 0. 1. ]]
M_R.dtype = float64
dist_R = [-0.44271713 0.01572439 0.01272985 -0.02997997 0.31224365]
dist_R.dtype = float64
R = [[ 0.99757046 0.06468998 -0.02585312]
[-0.06438798 0.99784853 0.01234882]
[ 0.02659635 -0.01065419 0.99958948]]
R.dtype = float64
T = [[-0.23299669]
[-3.05171467]
[ 0.43562338]]
T.dtype = float64
The results of stereoRectify using
size = image_L.shape[:2]
R_rectify_L, R_rectify_R, P_L, P_R, Q, validPixROI_L, validPixROI_R = cv.stereoRectify(M_L,
dist_L, M_R, dist_R, size, R, T)
P_L = [[ 4.30000000e+02 0.00000000e+00 2.68848224e+03 0.00000000e+00]
[ 0.00000000e+00 4.30000000e+02 -2.09073371e+03 0.00000000e+00]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
P_L.dtype = float64
P_R = [[ 4.30000000e+02 0.00000000e+00 2.68848224e+03 0.00000000e+00]
[ 0.00000000e+00 4.30000000e+02 -2.09073371e+03 -1.32932034e+03]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
P_R.dtype = float64
The values for pixel centers in P_L and P_R are factor 10 to large. Using these results as in
map1_L, map2_L = cv.initUndistortRectifyMap(M_L, dist_L,
R_rectify_L, P_L, size, cv.CV_32FC1)
image_rectified_L = cv.remap(image_L, map1_L, map2_L, cv.INTER_LINEAR)
map1_R, map2_R = cv.initUndistortRectifyMap(M_R, dist_R,
R_rectify_R, P_R, size, cv.CV_32FC1)
image_rectified_R = cv.remap(image_R, map1_R, map2_R, cv.INTER_LINEAR)
gives completely black images. However, if I separate the undistort step like:
map1_L, map2_L = cv.initUndistortRectifyMap(M_L, dist_L, np.eye(3),
M_L, size, cv.CV_32FC1)
image_interp_L = cv.remap(image_L, map1_L, map2_L, cv.INTER_LINEAR)
map1_R, map2_R = cv.initUndistortRectifyMap(M_R, dist_R, np.eye(3),
M_R, size, cv.CV_32FC1)
image_interp_R = cv.remap(image_R, map1_R, map2_R, cv.INTER_LINEAR)
and than use stereoRectify on the undistorted images as in:
R_rectify_L, R_rectify_R, P_L, P_R, Q, validPixROI_L, validPixROI_R = cv.stereoRectify(M_L,
np.zeros((1,5)), M_R, np.zeros((1,5)), size, R, T)
map1_L, map2_L = cv.initUndistortRectifyMap(M_L, np.zeros((1,5)),
R_rectify_L, P_L, size, cv.CV_32FC1)
image_rectified_L = cv.remap(image_interp_L, map1_L, map2_L, cv.INTER_LINEAR)
map1_R, map2_R = cv.initUndistortRectifyMap(M_R, np.zeros((1,5)),
R_rectify_R, P_R, size, cv.CV_32FC1)
image_rectified_R = cv.remap(image_interp_R, map1_R, map2_R, cv.INTER_LINEAR)
The images and resulting projection matrices look fine.
P_L = [[ 430. 0. 199.88677025 0. ]
[ 0. 430. 126.72800636 0. ]
[ 0. 0. 1. 0. ]]
P_L.dtype = float64
P_R = [[ 4.30000000e+02 0.00000000e+00 1.99886770e+02 0.00000000e+00]
[ 0.00000000e+00 4.30000000e+02 1.26728006e+02 -1.32932034e+03]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00 0.00000000e+00]]
P_R.dtype = float64
ps: the images are acutally top and bottom and not left and right.