Ask Your Question

Stereo rectification is not row aligned

asked 2015-01-08 11:01:36 -0500

thomas314 gravatar image

I try to rectify two images taken with a Samsung Galaxy S to calculate a depth map. They seem to be correctly rectified, but they are not row aligned, that is the drawn epipolar lines are horizontal, but have different y-values. I rectify the images with stereoRectify(), call initUndistortRectifyMap/remap to rectify the two images.

The rotation/translation is calculated with the fundamental matrix (using manually specified correspondences). It is converted to the essential matrix using intrinsic parameters. Then rotation/translation can be calculated using the formulas specified in Hartley&Zisserman's Multiple View Geometry, "9.6.2 Extraction of cameras from the essential matrix", p. 258

Question: Can you give me a hint to align the rows? In the few tutorials/examples that I found, the images are row-aligned after stereoRectify/initUndistortRectifyMap/remap

I calibrated the camera using a 6x8 checkerboard pattern. The colored lines represent the epipolar lines

rectified images

If you want to rectify the images by yourself:

/* to view the two rectified images side by side,
   insert this source code in samples/cpp/stereo_match.cpp, after the following lines:
img1 = img1r;
img2 = img2r;

cv::Mat imgboth(img1.rows, img1.cols*2, img1.type());
img1.copyTo(cv::Mat(imgboth, cv::Rect(0,0, img1.cols, img1.rows)));
img2.copyTo(cv::Mat(imgboth, cv::Rect(img1.cols, 0, img2.cols, img2.rows)));

cv::namedWindow("rectified", cv::WINDOW_NORMAL);
cv::imshow("rectified", imgboth);
while(cv::waitKey(0) != 'q');

Here the intrinsic and extrinsic parameters.

Here the original images for comparison: screen0.jpg screen1.jpg

edit retag flag offensive close merge delete


thank you! I already thought I made some mistakes. Can you post your comment as an answer, so that I can accept it?

thomas314 gravatar imagethomas314 ( 2015-01-09 05:08:50 -0500 )edit

2 answers

Sort by ยป oldest newest most voted

answered 2015-01-09 03:33:44 -0500

The reason why they are row aligned in samples is because the setup assumes that your baseline of your camera's have a common y coordinate, meaning they are on the same level compared to eachother which is common practice in a stereo setup. Because a smartphone moves also in y direction you will have to individually apply a translation to the image based on the lines. Match color levels on y coordinate and calculate the average transformation needed.

edit flag offensive delete link more


There you go!

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-09 06:25:50 -0500 )edit

answered 2015-01-22 03:14:13 -0500

thomas314 gravatar image

updated 2015-01-22 05:22:07 -0500

I implemented a stereo rectification algorithm based on "A compact algorithm for rectification of stereo pairs" by Fusiello et al. It works quite well for my use case.

But I found out what I did wrong: check the camera calibration matrix!

I checked the cv::stereoRectify again, with a generic camera calibration matrix [2600, 0, 1280 0, 2600, 960 0, 0, 1] and the distortion parameters found by calibration. Now I get this result:

image description

edit flag offensive delete link more



Could you share your implementation maybe for future users?

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-22 03:16:05 -0500 )edit

I checked the cv::stereoRectify again, with a generic camera calibration matrix [2600, 0, 1280 0, 2600, 960 0, 0, 1] and the distortion parameters found by calibration.

Now it's almost perfectly aligned. So my camera calibration was wrong ...

thomas314 gravatar imagethomas314 ( 2015-01-22 05:17:28 -0500 )edit

Okay, glad to see it solved!

StevenPuttemans gravatar imageStevenPuttemans ( 2015-01-22 06:36:49 -0500 )edit

For me stereoRectify still is not properly aligned, and stereoRectifyUncalibrated outputs nonsense. @thomas314 would you mind sharing your full solution?

temp9412 gravatar imagetemp9412 ( 2015-06-07 03:17:45 -0500 )edit

HELLO Thomas, I am very intrested in image rectificaion, my background is Survying Engineering. I am wondering if it is possible to have your code, beacsue I want to rectify stereo pair of images, and the interior and extror parametres are known. My email is Thanks in advance.

Haval gravatar imageHaval ( 2017-02-28 14:17:36 -0500 )edit
Login/Signup to Answer

Question Tools



Asked: 2015-01-08 11:01:36 -0500

Seen: 2,397 times

Last updated: Jan 22 '15