Having trouble with stereoRectify

asked 2013-09-16 18:46:37 -0600

thevidyy
21 ●1 ●4

updated 2013-09-17 06:38:27 -0600

jensenb
730 ●11 ●18

I ran Bouguet's calibration toolbox (http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/example.html) on Matlab and have the parameters from the calibration (intrinsic [focal lengths and principal point offsets] and extrinsic [rotation and translations of the checkerboard with respect to the camera]). Feature coordinate points of the checkerboard on my images are also known. I want to obtain rectified images so that I can make a disparity map (for which I have the code for) from each pair of rectified images.

Here is my code that keeps breaking off at the stereoRectify function, saying that there was an "unhandled exception": https://gist.github.com/anonymous/6586653

It might be worthy to note that I did not use stereo cameras. It was just one camera, and images were taken with the camera moving relative to the scene. Is stereoRectify still applicable here?

edit retag flag offensive close merge delete

add a comment

answered 2013-09-17 06:37:20 -0600

jensenb
730 ●11 ●18

updated 2013-09-18 15:29:21 -0600

Skimming over your code I did not notice anything immediately wrong. I did notice however that your rotation between the two camera poses seems a bit too large. Have you tried doing stereo rectification with Bouguet's toolbox? The OpenCV implementation is based upon his Matlab implemenation. If you also have problems with the Matlab version, I suspect that the rotation between the cameras may be too large, causing numerical problems.

One quick way to check is calculate the location of the epipoles in both the camera coordinate systems (these are the null spaces of the fundamental matrix). If the epipole's are near or within your image borders, then the rotation is too large the OpenCV rectification algorithm likely won't work. In this case you can either modify your camera setup, or you can use alternative rectification algorithms, such Pollefey's polar algorithm (M. Pollefeys, R. Koch and L. Van Gool, A simple and efficient rectification method for general motion, Proc. International Conference on Computer Vision).

Edit: So I have found the immediate error in the code gist from the question. The OpenCV calibration and rectification code likes doubles, and you were using floats, causing the error message you have received. I have forked an updated version here: gist.

Nonetheless, like I said above, your calibration result indicates a somewhat suboptimal stereo setup, so don't expect great rectification results. With your parameters, stereoRectify completely rotated the right image out of view, while zooming and almost completely cropping the left image. This can best (and most easily) be solved by making changes to your stereo setup. The alternative is to write your own rectification code (I went this route due to my somewhat extreme lens distortion).

edit flag offensive delete link

Comments

How did you figure that the rotations were too large by just looking at it? Just to clarify, the inputs R and T are the rotations and translations between the two stereo camera reference frames, right? The rotations and translations outputted by Bouguet's toolbox were with respect to the camera reference frame, I assumed, so using these parameters, I derived R and T between the two stereo camera frames (which might be the source of error).

Here is what I derived:

R = (Rc_2)(Rc_1^T)

T = Tc_1 - (R^T)(Tc_2),

where Rc, Tc are the rotations and translations of the camera with respect to the scene.

The rectification function in the toolbox could not run - it kept running out of memory since my images were too large, hence the reason I'm trying to do the rectification on opencv.

thevidyy ( 2013-09-17 18:46:20 -0600 )edit

It might not be the poses that's the problem here... OpenCV is giving the error that "the formats of input arguments do not match in unknown function," which is weird because all of my matrices M, D, R, and T have the data type CV_32FC1. Do you know what's going on here? I'm not too familiar with the data types on opencv.

Also, it's calling stereoRectify an unknown function. Is it possible that I'm just missing the function?

thevidyy ( 2013-09-17 20:27:49 -0600 )edit

I didn't mean to say that the rotation is too big and that is why it fails, just that might be a reason to consider. In my stereo setup for example, the rotation between both camera frames is close to the identity.

I don't understand how you used the stereo calibration function of the matlab toolbox. Why do you have rotations (Rc) and translations (Tc) with respect to some scene coordinate? When I used the toolbox last, the left camera extrinsics was set to world coordinates origin ( [I | 0]), and the output of the calibration was a rotation vector and translation vector from the right camera into the first. For stereo rectification, there is no benefit to using the full extrinsic parameters of both cameras.

jensenb ( 2013-09-18 13:24:36 -0600 )edit

add a comment

Having trouble with stereoRectify

1 answer

Comments

Links

Question Tools

Stats

Related questions

Having trouble with stereoRectify edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

Having trouble with stereoRectify