# Is Toe-in fatal for Stereo Correspondence?

I have had some success using pairs of webcams for stereo vision. More recently, I have tried using consumer 3D cameras for the same application, and I am having trouble with them.

It appears that the lenses are not exactly parallel and that they are "cross-eyed" enough that more-distant features sometimes have less disparity than closer features. In other words, the two views converge something in the middle distance has zero disparity.

This would seem to be an insurmountable problem. Are there any known-good ways to deal with it?

edit retag close merge delete

Sort by » oldest newest most voted

The short answer is "no," it is not fatal. One thing you can do is set minDisparity to a negative number, which seems to work for the stereoBM and stereoSGBM functions. But you may not even have to do that, because the calibration/rectification process can offset the images so that the correspondence will never see negative disparities.

So, my troubles must come from elsewhere, alas.

more

I was into a similar problem (by the way, why is it called "toe-in"?). What I had to do was to get a disparity map from a stereo pair taken with two "potentially" converging cameras. I say "potentially" because the converging factor was variable from pair to pair.

I had to modify my own stereo matching algorithm from a simple "right-to-left" correspondence search to a "right-to-left & left-to-right" method.

In few words, if your method does that:

for disp = 0 to MAX_DISPARITY
score = correspondence between left(x ; y) and right(x - disp ; y)
if score is better than the previous
disparity(x ; y) = disp


you have to modify it this way

for disp = 0 to MAX_DISPARITY
score1 = correspondence between left(x ; y) and right(x - disp ; y)
// move also to the right
score2 = correspondence between left(x ; y) and right(x + disp ; y)
score = max(score1, score1)
if score is better than the previous
if score1 > score2
disparity(x ; y) = +disp
else
disparity(x ; y) = -disp


In this way, you will save a disparity map where values from -MAX to 0 are referring to the object furthest than the converging point, and 0 to MAX referring to the ones closer to the observer.

The problems of this algorithm are basically two:

1. There are more operations to do
2. You need a robust correspondence metric, because it grows the numer of false positives
more

That method might work to get correct disparities, but the disparities it gets will only ambiguously suggest distances. For example, if a feature has a disparity of 5, it could be 5 units in front of, or 5 units behind, the zero-disparity plane.

If I understand you comment, nope, they aren't ambiguous. For a disparity value of "+5", the object is tretated to be on the front, while for a value of "-5" (negative!) it's behind the zero value. The important fact is that now the disparity can be positive but also negative

Official site

GitHub

Wiki

Documentation