Inpainting, normal behavior or a bug?

asked 2016-02-04 04:37:04 -0600

4133 ●12 ●41 ●101

updated 2016-02-04 10:09:00 -0600

Well as the title says I want to smooth a depth map which has blank spots (black spots where there is no depth information). Since this is a problem that other people have faced before I found this solution which works quite nice and it is based on inpainting functionality. However, after applying inpaint on the input image there are still have some black spots on the borders. My first thought was that this might have to do something with the used inpainting method (method by Alexandru Telea INPAINT_TELEA), but changing it to the Navier-Stokes INPAINT_NS based method didn't change the result. I do not really get what is happening and if it is a bug in the inpaint() code or something that I should expect. Any ideas?

Input image:

image description

Mask of no depth information pixels:

image description

Output with the problem (notice the black artifacts on the left and top borders):

image description

code i used:

    Mat depth;
    const unsigned char noDepth = 0; // change to 255, if values no depth uses max value or use the mask image
    cv::inpaint(img, (img == noDepth), depth, 5.0, INPAINT_TELEA); // img is the 8-bit input image (depth map with blank spots) you can just pick it from the above images if you want to replicate the issue

Update:

@berak well I used the xphoto::inpaint() that you suggested but though I do not get the black spot artifacts in the borders the quality of the image is not that good/smooth compared to the normal inpaint() as you can see:

image description

I do not know if I can modify any parameters in order to get a better result but as far as I looked there is not such an option. Therefore, I think I will stick to the normal inpaint()hoping to get an answer about what is causing the artifacts in the borders.

edit retag flag offensive close merge delete

Comments

did you try making a border around your image (say, a few pixels), so inpaint can kinda "wash out" there ?

(have not tried myself, just a blunt idea)

berak ( 2016-02-04 05:53:58 -0600 )edit

@berak nope I haven't tried that. But what pixel values should I use, for the make up border? Moreover, do you think that this behavior from inpaint() is a bug in the implementation code of the function? I was thinking to create an issue at github.

theodore ( 2016-02-04 06:53:31 -0600 )edit

Maybe inpaint() needs full context to work? Meaning, a black blob should have valid pixels all around ?

Pedro Batista ( 2016-02-04 09:47:30 -0600 )edit

@Pedro Batista indeed this could be the case. On the other hand looking on the result by using xphoto module's version there isn't such a problem, of course it uses another method. To be honest I do not know, for that reason I am asking here. I think it would be a nice idea to open an issue at github and see what the devs would say, what do you think?

theodore ( 2016-02-04 10:07:28 -0600 )edit

@theodore, just dare to make an issue. they won't bite ;)

berak ( 2016-02-04 10:11:04 -0600 )edit

@berak I will ;-). By the way you were right about making a border around the image. I will add the workaround in a while. I do not know though if it can be considered as a solution.

theodore ( 2016-02-04 10:30:51 -0600 )edit

@berak Can you please help me here? link text

Ayesha Siddique ( 2016-02-06 02:34:04 -0600 )edit

add a comment

answered 2016-02-04 11:06:06 -0600

theodore

4133 ●12 ●41 ●101

Ok, taking into account the fact that normal inpaint() cannot handle borders, @berak 's suggestion and the solution from @Micka at SO here a workaround could be the following:

replicate border pixels using copyMakeBorder()
apply inpaint(), your borders will still have problem though
extract the original area without border, borders information having aritfacts is excluded

code:

// add 1pixel border
copyMakeBorder(img, img, 1, 1, 1, 1, BORDER_REPLICATE); 
const unsigned char noDepth = 0; // change to 255, if values no depth uses max value or use mask image
// apply inpainting
cv::inpaint(img, (img == noDepth), img, 5.0, INPAINT_TELEA);
// extract the original area without border:
cv::Mat new_img = img(cv::Rect(1, 1, img.cols-1, img.rows-1));

output:

image description

However, this does not answer if the initial behavior of inpaint() is expected or it is based on a bug.

edit flag offensive delete link

Comments

As far as I understand inpainting, it seems that a center pixel value is defined by all the pixels around it. If you have the border and some of those edge pixels have surrounding pixels falling outside the image, means that accessing them would result into illegal memory access. Therefore I am guessing there is an ignore region at the border.

StevenPuttemans ( 2016-02-05 04:59:32 -0600 )edit

yes I agree, but this should be the case for the right and bottom borders, shouldn't be?

theodore ( 2016-02-05 06:05:38 -0600 )edit

hmm true :D Might be that they tried to fix it, but the solution is buggy :D?

StevenPuttemans ( 2016-02-05 07:06:03 -0600 )edit

let's see I opened an issue here

theodore ( 2016-02-05 07:33:21 -0600 )edit

add a comment

Inpainting, normal behavior or a bug?

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Inpainting, normal behavior or a bug? edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Inpainting, normal behavior or a bug?