# Depth Discontinuities in GPU::BM and GPU::CSBP

I am interested in using the stereo correspondence algorithms in the GPU module (2.4.X). However, the results I am able to achieve are not useable as there are great errors at depth edges/discontinuities. My question is this: is there a way to achieve quality segmentation (sharp edges) at severe depth discontinuities in either the gpu::BM or gpu::CSBP? I know from reading the paper on CSBP that this is a problem inherent to the method and is supposedly fixed by using a bilateral or median filter.

How have you gotten the best results from the real-time parallel stereo methods (not BP)? How did you eliminate poor matches? What postprocessing steps did you follow?

This is the disparity image I generate with CSBP (and some first attempts at filtering). Notice the absence of a depth 'shadow' where the disparity really shouldn't be estimated:

For comparison, see the quality of my results with SGBM:

edit retag close merge delete

Hi: Were you ever able to resolve this issue?

( 2014-02-21 19:21:47 -0500 )edit
1

I think the solution would be to run the algorithm twice, matching once L to R and once R to L and then do a consistency check to eliminate bad matches/values. This would double the runtime, but is certainly worth a try.

( 2014-03-19 12:01:29 -0500 )edit

Sort by » oldest newest most voted

Here is code to do L-R, R-L, and a consistency check for CSBP.

d_left.upload(imgL);

// L-R disparity
scsbp(d_left, d_right, d_dispPre_LR);
dbf(d_dispPre_LR, d_left, d_disp_LR);

// Flip horizontally (1) for R-L disparity computation
cv::gpu::flip(d_left, d_left_f, 1);
cv::gpu::flip(d_right, d_right_f, 1);

// R-L disparity
scsbp(d_right_f, d_left_f, d_dispPre_RL_f);
dbf(d_dispPre_RL_f, d_right_f, d_disp_RL_f);

cv::flip(disp_RL_f, disp_RL, 1); // flip back (not supported on GPU)

//////////////////////////////////////////////////////////////////////
// Consistency Check:
//////////////////////////////////////////////////////////////////////
short* dLPtr = (short*) disp_LR.data;
short* dRPtr = (short*) disp_RL.data;
int cstep = disp_LR.cols;

for (int i = 0; i < disp_LR.rows; i++) {
for (int j = 0; j < disp_LR.cols; j++) {
short l_d = dLPtr[(i*cstep)+j];
short r_d = dRPtr[(i*cstep)+j-l_d];
if (abs(l_d-r_d) > 1) {
// set disparity to 0
dLPtr[(i*cstep)+j] = 0;
}
}
}

more

Hi, I got better results with the RL version when compared with LR, but when I try to fix the disparity as you suggested, the image becomes almost black. Could you comment why that conditional, is the objective to zero out displacement when its found in LR but not in RL? Thank you.

( 2015-02-18 19:08:02 -0500 )edit
1

The conditional is the consistency check. When the two disparity images don't agree, we conclude that we can't trust either of them. Right now it is set to 1, but perhaps you could change that to 2 and take the average (which should be included above anyway). Unfortunately this method (CSBP) only computes integer disparities and so you can't get any finer (than 0.5 through averaging). CSBP returns a dense disparity, so there is a value for every pixel, it is always 'found'.

( 2015-02-27 19:05:33 -0500 )edit

Official site

GitHub

Wiki

Documentation