Possible code bug in frame blending in gpu/NPP_staging.cu

asked 2014-02-25 21:07:32 -0500

updated 2014-02-26 05:09:22 -0500

berak gravatar image

Code of OpenCV 2.4.8:

I was trying to figure out how OpenCV doing the frame interpolation given forward and backward optical flow. I found the code in modules/gpu/src/nvidia/NPP_staging/NPP_staging.cu is very weird. Specifically, in function nppiStInterpolateFrames (Line 2006~2097) it is first to warp the flow fields (to offset the reference pixel position for doing inverse warping, in my opinion). However, in the 4th call of function nppiStVectorWarp_PSF2x2_32f_C1(), it is using bwdU as the output buffer (Line 2079), where I guess it should be bwdV instead. Checking the previous 3 call of the same function would easily discover that this should be a typo.

Another bug is inside function BlendFramesKernel, I believe Line 1936~1941, which is

if (b0 && b1)
{
    // pixel is visible on both frames
    out[pos] = tex2D(tex_src0, x - _u * theta, y - _v * theta) * (1.0f - theta) +
        tex2D(tex_src1, x + _u * (1.0f - theta), y + _v * (1.0f - theta)) * theta;
}

should be as follows instead (because it is supposed to use bi-directional optical flow),

if (b0 && b1)
{
    // pixel is visible on both frames
    out[pos] = tex2D(tex_src0, x - _u * theta, y - _v * theta) * (1.0f - theta) +
        tex2D(tex_src1, x - _ur * (1.0f - theta), y - _vr * (1.0f - theta)) * theta;
}

In practice, the two bugs are usually unnoticeable. But fixing the bugs does reduce artifacts in some cases. I suggest the authors check the code carefully and fix the obvious typo/logical errors. Thanks. Any comment is welcome.

edit retag flag offensive close merge delete

Comments

great !

now report your findings on the issues page

better even, if you can come up with a pull request on github to fix it .

berak gravatar imageberak ( 2014-02-26 05:14:11 -0500 )edit