Ask Your Question
0

How can i fill the gaps in the foreground and improve foreground extraction?

asked 2018-02-15 09:30:01 -0600

generator gravatar image

updated 2018-02-15 09:30:16 -0600

I want to improve my foreground extraction on my opencv.js project.I am extracting moving foreground from static background.My output foreground have gaps and little bit noisy. How can i fill this gaps with original foreground object color?

Input

Input

Output

Output

Here is my code:

let video = document.getElementById('videoInput');
let cap = new cv.VideoCapture(video);

let frame = new cv.Mat(video.height, video.width, cv.CV_8UC4);
let fgmask = new cv.Mat(video.height, video.width, cv.CV_8UC1);
let fgbg = new cv.BackgroundSubtractorMOG2(500, 16, false);

const FPS = 30;
function processVideo() {
    try {
        if (!streaming) {
            // clean and stop.
            frame.delete(); fgmask.delete(); fgbg.delete();
            return;
        }
        let begin = Date.now();
        // start processing.
        cap.read(frame);
        fgbg.apply(frame, fgmask);

        frame.copyTo(fgmask, fgmask)    
        cv.imshow('canvasOutput', fgmask);
        // schedule the next one.
        let delay = 1000/FPS - (Date.now() - begin);
        setTimeout(processVideo, delay);
    } catch (err) {
        utils.printError(err);
    }
};

// schedule the first one.
setTimeout(processVideo, 0);
edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
2

answered 2018-02-15 10:41:08 -0600

Tetragramm gravatar image

Take a look at this tutorial: Morphological Operations

edit flag offensive delete link more

Comments

Thank you.I am trying to Opening process because it says "It is useful in removing noise." but it doesn't work.

Without opening: https://streamable.com/xh368

With Opening: https://streamable.com/rq5tm

generator gravatar imagegenerator ( 2018-02-15 11:06:49 -0600 )edit

You need to read a bit more carefully. Notice how in the tutorial, Opening is white on a black background. In your case, you have (darker) things on a white background.

Tetragramm gravatar imageTetragramm ( 2018-02-15 11:14:02 -0600 )edit

Then,How can i implement Morphological Operations?Maybe "reversing background color and foreground color" work

generator gravatar imagegenerator ( 2018-02-15 11:39:27 -0600 )edit

Try copying the frame to a black image, using the fgmask.

Tetragramm gravatar imageTetragramm ( 2018-02-15 17:47:20 -0600 )edit

I use bitwise_not for reversing background color and foreground color.And i use opening for removing background noise and closing image for closing small holes inside the foreground objects.But It didn't work Without Closing and Opening : https://streamable.com/xh368

With Closing and Opening : https://streamable.com/bixmm

Closing and Opening Code : https://anotepad.com/notes/e2gbf4

generator gravatar imagegenerator ( 2018-02-16 03:07:30 -0600 )edit

So, you still aren't really reading what these functions are doing.

Take the code from your link and insert it before the frame.copyTo. Do not do the bitwise_not. That is because if you operate on the fgmask, you already have a white foreground and black background. It's after the copyTo that you get the white background.

Tetragramm gravatar imageTetragramm ( 2018-02-16 15:47:17 -0600 )edit
1

answered 2018-02-16 05:07:10 -0600

What you should do in my opinion

  • Apply on your binary mask some erosion and dilation until you only have pixels left that are part of the movement
  • Then you put all those points into a vector and calculate the convex hull on all those points
  • Now on top of your mask draw that convexHull with filled option on

This will include all the inner pixels also in your foreground mask!

edit flag offensive delete link more
-1

answered 2018-02-21 12:04:03 -0600

ManuVISION gravatar image
  1. A morphological open with a small kernel to remove all small speckles

  2. Find the largest closed contour and fit a convex hull around it.

  3. Fill the convex hull with white, while the background is black

  4. Use the binary mask from step 3 and copy the contents of the original image onto an all white image using the binary mask as reference.
    Original_Image.copyTo(All_White_Image,Binary_Mask)

edit flag offensive delete link more

Comments

Wait a second, what is different in your approach opposed to mine?

StevenPuttemans gravatar imageStevenPuttemans ( 2018-02-22 03:50:33 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2018-02-15 09:30:01 -0600

Seen: 2,096 times

Last updated: Feb 21 '18