Ask Your Question
1

Depth Discontinuities in GPU::BM and GPU::CSBP

asked 2013-11-22 16:47:07 -0500

updated 2015-03-01 11:30:09 -0500

I am interested in using the stereo correspondence algorithms in the GPU module (2.4.X). However, the results I am able to achieve are not useable as there are great errors at depth edges/discontinuities. My question is this: is there a way to achieve quality segmentation (sharp edges) at severe depth discontinuities in either the gpu::BM or gpu::CSBP? I know from reading the paper on CSBP that this is a problem inherent to the method and is supposedly fixed by using a bilateral or median filter.

How have you gotten the best results from the real-time parallel stereo methods (not BP)? How did you eliminate poor matches? What postprocessing steps did you follow?

This is the disparity image I generate with CSBP (and some first attempts at filtering). Notice the absence of a depth 'shadow' where the disparity really shouldn't be estimated:

image description

For comparison, see the quality of my results with SGBM:

image description

edit retag flag offensive close merge delete

Comments

Hi: Were you ever able to resolve this issue?

Shawarma89 gravatar imageShawarma89 ( 2014-02-21 19:21:47 -0500 )edit
1

I think the solution would be to run the algorithm twice, matching once L to R and once R to L and then do a consistency check to eliminate bad matches/values. This would double the runtime, but is certainly worth a try.

Der Luftmensch gravatar imageDer Luftmensch ( 2014-03-19 12:01:29 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2015-01-11 09:17:27 -0500

updated 2015-01-11 09:18:35 -0500

Here is code to do L-R, R-L, and a consistency check for CSBP.

d_left.upload(imgL);
d_right.upload(imgR);   

// L-R disparity
scsbp(d_left, d_right, d_dispPre_LR);
dbf(d_dispPre_LR, d_left, d_disp_LR);
d_disp_LR.download(disp_LR);

// Flip horizontally (1) for R-L disparity computation
cv::gpu::flip(d_left, d_left_f, 1);
cv::gpu::flip(d_right, d_right_f, 1);

// R-L disparity
scsbp(d_right_f, d_left_f, d_dispPre_RL_f);
dbf(d_dispPre_RL_f, d_right_f, d_disp_RL_f);
d_disp_RL_f.download(disp_RL_f);

cv::flip(disp_RL_f, disp_RL, 1); // flip back (not supported on GPU)

//////////////////////////////////////////////////////////////////////
// Consistency Check:
//////////////////////////////////////////////////////////////////////
short* dLPtr = (short*) disp_LR.data;
short* dRPtr = (short*) disp_RL.data;
int cstep = disp_LR.cols;

for (int i = 0; i < disp_LR.rows; i++) {
    for (int j = 0; j < disp_LR.cols; j++) {
        short l_d = dLPtr[(i*cstep)+j];
        short r_d = dRPtr[(i*cstep)+j-l_d];
        if (abs(l_d-r_d) > 1) {         
            // set disparity to 0
            dLPtr[(i*cstep)+j] = 0;
        } 
    }
}
edit flag offensive delete link more

Comments

Hi, I got better results with the RL version when compared with LR, but when I try to fix the disparity as you suggested, the image becomes almost black. Could you comment why that conditional, is the objective to zero out displacement when its found in LR but not in RL? Thank you.

Jean Mousinho gravatar imageJean Mousinho ( 2015-02-18 19:08:02 -0500 )edit
1

The conditional is the consistency check. When the two disparity images don't agree, we conclude that we can't trust either of them. Right now it is set to 1, but perhaps you could change that to 2 and take the average (which should be included above anyway). Unfortunately this method (CSBP) only computes integer disparities and so you can't get any finer (than 0.5 through averaging). CSBP returns a dense disparity, so there is a value for every pixel, it is always 'found'.

Der Luftmensch gravatar imageDer Luftmensch ( 2015-02-27 19:05:33 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2013-11-22 16:47:07 -0500

Seen: 1,360 times

Last updated: Mar 01 '15