Ask Your Question
1

SBM_Sample.cpp: Is this disparity map correct?

asked 2018-01-29 02:47:52 -0500

__Alex__ gravatar image

Hello, I ran the SBM_Sample.cpp with the rubberwhales, but the disparity map I got looked kind of erratic, see here

Is this really to correct map? I kind of expected something a lot smoother, judging from the two example input images rubberwhale1 and rubberwhale2. I use OpenCV 3.4.0 and I posted the code here, just for the sake of completeness.

Cheers Alex

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
2

answered 2018-01-29 18:47:46 -0500

updated 2018-01-29 18:48:34 -0500

The rubber whale images in samples/data/rubberwhale[12].png do not appear to be stereo camera left and right images. Instead, it appears as if one camera has stayed in the same location relative to many of the scene components, while selected elements in the image have been physically pushed or rolled slightly relative to others from one image to the other. This includes vertical movement of some components between the 1 and the 2 image.

This violates the stereo disparity calculation that the images are derived from two cameras viewing the same scene at the same time from different perspectives. Furthermore, the images need to be rectified - that is, isopolar - item features in the images need to be displayed on matching lines, even if they have been translated a different amount along the row due to being at a different distance from the camera's baseline.

Try again, using rectified left-right pairs, e.g. aloeL.jpg and aloeR.jpg.

edit flag offensive delete link more

Comments

1

Thanks for the hint! This map looks a lot better, see here. In that case I believe that the sample-code needs to be updated, because that code loads the rubberwhale images. I still have -16 as minvalue, though. Is that a default value or something? And is there a way to convert disparity to pixels?

__Alex__ gravatar image__Alex__ ( 2018-01-30 02:11:21 -0500 )edit
1

@__Alex__ - I did some playing around with the aloe image.

In imgDisparity16S, a non-negative array element value indicates the disparity match determined for that pixel. A value less than the minDisparity (in this case, -16, because minDisparity defaults to 0) indicates no valid disparity match determined for that pixel.

When the 16-bit signed image is converted to an 8-bit unsigned image for display, the negative values are interpreted as 0 and displayed as black.

With the aloe images, there is a wide range in disparities (that is, distances), and you'll get better coverage for the scene by setting ndisparities = 16*10.

I'm not sure why the rubberwhale images are the defaults, or why the program ignores any image file path parameters, yet says it wants them in the help message. Huh!

opalmirror gravatar imageopalmirror ( 2018-01-30 14:01:13 -0500 )edit
1

So, I think you're on to a few things here. You could even open an issue at github opencv issues .

  1. The default images should be changed from the rubberwhale images (optical flow test images) to rectified left and right images (e.g. aloeL.jpg and aloeR.jpg).
  2. An appropriate minDisparity and numDisparities should be updated to match the new default images.
  3. Program ignores its image parameters - if this is intentional, then remove mention of parameters. If it is not, then add and test support for them.
  4. Any references to the example should be found and updated, and if there's a test that runs this program, the test should be updated as well.
opalmirror gravatar imageopalmirror ( 2018-01-30 14:13:19 -0500 )edit
1

Forgot to mention, the disparities correspond 1:1 with pixels in the left image. The convertTo step in the program converts the 16-bit signed disparity values into a displayable 8-bit unsigned grayscale image with pixels in the range 0 (corresponding to minVal or 0, whichever is greater) up to 255 (corresponding to maxVal - minVal).

opalmirror gravatar imageopalmirror ( 2018-01-30 14:21:31 -0500 )edit
1

Okay, I opened an issue here. Thanks for explaining the disparities to me. My mind was so fixated on relative numbers that I didn't expect them to be absolute x-values. And also thanks for mentioning optical flow. I don't like having to perfectly rectify my pictures, because I'm not doing SBM to see 3D, I just need some feature-points matched, so maybe I'll abuse optical flow algorithms for that instead.

__Alex__ gravatar image__Alex__ ( 2018-01-31 02:23:38 -0500 )edit

PS: When I set ndisparities to 16x10 I get some disparities that exceed the image width of 1282, e.g. 2023. What does that mean?

__Alex__ gravatar image__Alex__ ( 2018-01-31 02:41:19 -0500 )edit
1

@__Alex__ Great first issue post on github, thanks!

The disparity output array, returned by argument from StereoBM::compute() (and in all classes derived from cv::StereoMatcher, uses fixed point arithmetic. The lowest cv::DISP_SHIFT (4) bits are the fractional part. This means that each increment of cv:DISP_SCALE (16) corresponds to 1 pixel match offset between left and right image.

In a calibrated stereo system, each disparity value corresponds inversely to a distance from an idealized left camera, using a coordinate system whose origin is at the left camera's center pixel.

Optical flow with key points is an excellent solution for many problems. It's less accurate, but lower computation, and no fussy fine calibration

opalmirror gravatar imageopalmirror ( 2018-01-31 13:23:19 -0500 )edit
1

Got it, I shall divide by 16 then, and re-test stereobm for a bit. And thanks for the compliment! :-)

__Alex__ gravatar image__Alex__ ( 2018-02-02 09:35:58 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-01-29 02:47:52 -0500

Seen: 539 times

Last updated: Jan 29 '18