StereoRectify - does it work

asked 2018-08-30 06:52:20 -0600

JT3D gravatar image

updated 2018-09-05 13:58:31 -0600

Hi all,

I have a stereo camera setup - about 100mm apart

Simply I would like to get the matrix R1 R2 P1 P2 and Q so that I can first create a disparity map and next a point cloud

I am using StereoCalibrate with the following flags and pass in the intrinsic information that I have got from separate calibrations of each camera


I get around 0.8 rms for stereoCalibrate ()

I then use StereoRectify to get R1 R2 P1 P2

When I use StereoRectify it doesn't matter what setting I put into this function (CALIB_ZERO_DISPARITY set / not set) and what I change alpha value (-1, 0, 1) too the images do not rectify properly.

Also when I look at the extrinsic values for R1 R2 P1 P2 they look totally incorrect -even for simple values like fx and fy even though they are passed into sterocalibrate correctly

Now when I use StereoRectifyUncalibrated it computes R1 R2 P1 P2 correctly and the images are rectified correctly

I tried running the same code on both 3.3.1 and 3.4.2 as there has been talk of something wrong with StereoRectify in 3.4.1.

Am I missing something or is there a bug with StereoRectify in version 3.4.2 and 3.3.1?? (I would prefer to use it over stereocalibrateUncalibrated)

Thanks in advance for your inputs


Update 1:

Hi all,

Attached is an image where I use StereoRectify. You can see the images are not row aligned.

When I use SterreoRectifyUncalibrated it works fine


I have tried this on the following versions

3.4.2 & 3.3.1

The problem I have looks very similar to this problem but I am not sure if this is still open or not

The code I am using is stereo_calib.cpp which is available from the examples folder in cpp in OpenCV

Any inputs would be very welcome.


update 2:

This is the calibration file I am using. You can see the R & T matrix in here along with the intrinsic values I am using. The cameras are 2448 x 2048 with a 12.5mm lens which should equate to 3623 for f (12.5mm / 0.00345(size of pixels in camera))

cam1_intrinsics: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 3.720140438796924e+03, 0., 1.285507273110709e+03, 0.,
       3.724303519202642e+03, 1.005642652331629e+03, 0., 0., 1. ]
cam1_distorsion: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data: [ -0.057043329274289, 0.223182551657098,
       0., 0.,
       0. ]
cam2_intrinsics: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 3.782786803634638e+03, 0., 1.241950747511028e+03, 0.,
       3.788182391361000e+03, 9.782339357588124e+02, 0., 0., 1. ]
cam2_distorsion: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data: [ -0.062491094405233, 0.252016985281970,
       0., 0.,
       0. ]
R: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 9.5956335394786729e-01, -4.0891553813976841e-02,
       -2.7850682323239140e-01, 3.9918591084260990e-02,
       9.9916089220343407e-01, -9.1661102483683431e-03,
       2.7864794247610625e-01, -2.3221364982101583e-03,
       9 ...
edit retag flag offensive close merge delete


Your question is still a bit vague to reproduce. Are you able to reproduce the misbehavior by compiling the stereo_calib.cpp or stereo_match.cpp sample source files, or cause the tests to fail? Otherwise, can you paste a short piece of reproducer code and a couple small sample images into your question which will show how it fails, with specific OpenCV versions and build environment it fails on?

opalmirror gravatar imageopalmirror ( 2018-08-30 14:37:02 -0600 )edit

Hi, I’m using stereo_calib.cpp as the basis for the testing. I’ll try and upload more detail on the problem with images etc as soon as I’m in front of my PC.

JT3D gravatar imageJT3D ( 2018-08-30 15:14:34 -0600 )edit

wait for your images

jsxyhelu gravatar imagejsxyhelu ( 2018-08-31 19:43:29 -0600 )edit

I edited my main question and added an image showing the incorrect rectification. Do you need more images to comment? Thanks

JT3D gravatar imageJT3D ( 2018-08-31 20:39:24 -0600 )edit

@JT3D have you check estimated R and T are correct. Can you put that matrix over here ?

ak1 gravatar imageak1 ( 2018-09-05 01:21:46 -0600 )edit

Hi, I edited the question again (Update 3) to add the calibration file for both intrinsic and extrinsic values. If required I could upload the calibration images for each stereo pair but I'm not sure if someone will invest that sort of time trying to figure out what is going wrong. Thanks in advance

JT3D gravatar imageJT3D ( 2018-09-05 03:12:28 -0600 )edit

Hi, have you found the answer for this. I am having exactly the same issue. With StereoRectify, I got the matrices R1,R2, P1, P2, and then using in initUndistortRectifyMap(), remap(), the rectification is totally off. However, if using stereoRectifyUncalibrated, the rectification is perfectly right.

Phuong Do gravatar imagePhuong Do ( 2019-10-02 07:29:06 -0600 )edit