Inconsistent Disparity Map Coloration Between Frames.

edit

You are seeing calculated depths, which, frame to frame, are calculated as very different disparities from your stereo cameras. Some have the appearance of a moving pattern of depth changes across the scene.

Your depth calculation problems are a direct result of several things:

1. Pulsed lighting
2. Short exposure
3. Uncoordinated shutters
4. Large low contrast areas in scene

Pulsed lighting and short exposure combine to create a problem where each camera picks up a different overall scene illumination intensity. Also, frame to frame, images from an individual camera's scene also vary wildly in intensity.

To correct the pulsed lighting and short exposure problem, change the lighting to constant, e.g. incandescent, so that the lighting intensity level is constant relative to a camera frame period. Alternatively, increase the exposure time to greater than the period of the lighting pulse period - typically, an exposure of 30ms or so is generally long enough to smooth these out - with a corresponding decrease in aperture, ISO, or gain setting, so that the images aren't blown out.

Uncoordinated shutters cause multiple problems. If they are offset in time but have the same frame period, then the two cameras (with the pulsed lighting and short exposure time) have a much different scene intensity. In the disparity algorithm in low contrast areas especially (more on this in a moment), this intensity offset is calculated as a difference in depth from the camera, resulting in a lot of the massive frame to frame fluctuation in calculated depthin your video. As an aside, another problem with uncoordinated shutters, is that a moving object captured at different times by left and right cameras will have a different depth calculation, depending on direction and speed of movement left or right.

The second major uncoordinated shutter problem is if the two cameras are running on internal frame timers; one may be taking exposures at a rate that is a few percentage different from the other. This will create a periodic pattern to the depth fluctuations already intrinsic in the short exposure and uncoordinated shutter problem. I think that is what you see with the pattern of disparity errors repeating about every second or so - it's a multiple beat frequency effect of the two out of sync frame timers and the lighting intensity changes.

To coordinate frame start times, the exposures of the cameras should be slaved to a common externally generated trigger signal. With these changes, any pulsed lighting intensity change will become much less significant frame-to-frame.

Large low contrast areas in the scene, means big areas of the scene (larger than the disparity comparison block size) have no high-contrast spots or edges. Without high-contrast spots or edges in the image which are unique, crisply located, and easy for the algorithm to identify the common feature in both cameras, the disparity algorithm has to resort to a local intensity comparison between the left and right frames to compute disparity. With short exposure pulsed lighting uncoordinated shutters, this is a recipe for ... (more)