Ask Your Question

problem with stereoRectify() results [magnified output]

asked 2019-04-16 11:00:10 -0600

3cc3bc3b1 gravatar image

updated 2019-04-23 22:35:02 -0600

supra56 gravatar image

Hello, newbie here. I have problems concerning the rectification of a stereo image pair. I have as a input videos from a pair of cameras attached to a stereo microscope. My final task is to 3D reconstruct the result. At this point I have stereo calibrated them with a reprojection error of about 1.1 per camera and 4 for the stereo result. After, I pass the calibration results for stereo rectification in stereoRectify() and remap.

stereoRectify(KL , DL , KR , DR , image_size , R , T , R1 , R2 , P1 , P2 , Q , 0, 0.01 ) ;

initUndistortRectifyMap(KL,DL,R1,P1,image_size, CV_16SC2 , map11, map12) ;
initUndistortRectifyMap(KR,DR,R2,P2,image_size, CV_16SC2 , map21, map22) ;

remap(initial_image[0],remappedImage0, map11,map12, INTER_LINEAR, BORDER_CONSTANT, Scalar());
remap(initial_image[1],remappedImage1, map21,map22, INTER_LINEAR, BORDER_CONSTANT, Scalar());

Now the output image pair looks rectified, but its a scaled up version of the input. Actually I see only a magnified region of the center on both final images as the rectification result.

I try to realize why this is happening, but I'm still missing something and I'm stuck. Is there any hint or reference or someone that has faced similar problems in the past? I attach a couple of images for reference. One is with the pair of the original and the rectified image (where the zoom in is obvious), and the second one is the rectification result.

Any help would be invaluable. My opencv version is 3.4.5 on Linux Mint 18.3

-A pair of the original and the rectified image

A pair of the original and the rectified image

  • Stereo rectification image pair

Stereo rectification image pair

edit retag flag offensive close merge delete


The stretching along the Y axis makes me think your calibration might have misdetected values stored in a K matrix (fx, fy, cx, cy), or come up with some invalid distortion matrix values. Could you include left and right original images, left and right K and D matrices, right R and T matrices? Also, discuss your procedure and tools for both mono (left and right K and D) and stereo (right R and T) calibration. We can probably get to the bottom of this.

opalmirror gravatar imageopalmirror ( 2019-04-17 13:27:22 -0600 )edit

Hello, first of all, thank you for the answer. When I tried to reproduce my calibration results, I found a error score near 1.2 for each camera but a big stereo calibration error in the scale around 13-14 (which I didn't notice at first). So now I took new videos to test my calibration again and after I will post what happened, if the rectification result was clearly an error of the calibration procedure or if the problem persists. Best regards

3cc3bc3b1 gravatar image3cc3bc3b1 ( 2019-04-18 04:38:02 -0600 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2019-04-21 11:12:21 -0600

3cc3bc3b1 gravatar image

updated 2019-04-21 11:33:19 -0600

I follow the example of Kaehler and Bradsky book (Learning Opencv3 chapter 19), to get more familiar with the procedure and the techniques because I have a lack of experience. First, I use as calibration targets a checkerboard of 7x10 squares size and also an asymmetric dots patern of 4x11 size. These targets are really small (the side of each square of the checkerboard is 0.45 mm). Each asymmetric dot has a size of about 0.25mm. I achieved the best results with asymmetric dots pattern, with the best result used for the stereorectification process below.

I use cv::calibrateCamera at each camera and after I use stereoCalibrate for the next step. I collect at least 100 successful measurements for a valid result. Then the stereo rectification follows. I use Bouguet method for rectification and pass my calibration results to cv::stereoRectify. After I apply cv::initUndistortRectifyMap to each camera frame and finally remap. At the end I store the results as the ones that uploaded here.

I acquired a better error score for my calibrations after shooting new calibration videos with better light control and being careful to present the targets on the whole field of view. With 135 detected stereo pairs, the reprojection error for stereoCalibrate was 3.3062141238092657e+00 . The error for the left camera is 6.0338782277157943e-01 and for the right camera similarly 6.0971222421947813e-01. The flag I used for stereo calibration was only CALIB_FIX_INTRINSIC, but I didn't see any difference with CALIB_FIX_ASPECT_RATIO | CALIB_SAME_FOCAL_LENGTH. The error score of ~ 0.6 per camera and 3.3 for stereo is better than my previous results. So after running the programm I see that I have a way better than the previous result, but I'm not quite sure that I can assess it correctly as adequate because my final aim is the 3d reconstruction. I run stereoRectify() with alpha set to -1 otherwise it produces quite funny results which doesn't resemble at all my initial frames (strips or discs of image parts). I also put flags=0 for stereoRectify.

Now the new calibration results are: Pattern: Asymmetric dots.


5.7432741355633716e+03 0. 3.5864063009366998e+02 0.
5.3294883722077411e+03 2.8626995097397503e+02 0. 0. 1.


5.5855914241748869e+03 0. 3.5893764153026808e+02 0.
5.2368003077143685e+03 2.8626284627542736e+02 0. 0. 1.


    2.7743096670749008e+00 -2.1332159140099390e+02
-2.3919561436865893e-02 -5.6926044040429592e-02


2.4194267392305364e+00 -1.5408901350984709e+02
-1.5277247740632098e-02 -5.7579530702796311e-02


    9.9937260710853593e-01 -2.1629178382018797e-02
2.8045869635414729e-02 2.1973416854983850e-02 9.9968624039129861e-01
-1.2024546728580759e-02 -2.7776988908176835e-02
1.2633266197999683e-02 9.9953431130320158e-01


-1.7948023556461674e+00 7.6656265194349504e-01


7.2820204955379997e-03 1.3097030873873605e+00 7.5056862812901337e-01
-1.3494652453095344e+00 5.0801381730943410e-02
1.7574949326414129e+00 -8.0551965631772415e-01
-1.7776590988214844e+00 8.2768590249103491e-05


1.3686591816576201e-07 2.6527131904819632e-05 7.3377205091950545e-02
-2.7052554056256310e-05 1.0974787539235105e-06
2.1173656674781893e-01 -7.6869609851764387e-02
-2.1094641314552989e-01 1.

I suppose that microscope calibration and rectification poses bigger challenge because like FOV, depth of ... (more)

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2019-04-16 11:00:10 -0600

Seen: 2,557 times

Last updated: Apr 23 '19