Improve background segmentation through BackgroundSubtractMOG2

edit

Hi,

I have created a small program that is supposed to detect changes in the foreground over a background. It works, but not as well as I would like it to. Here is what I have done so far, to try to improve it on my own.

• Tweaked parameters (such as history length, learningRate, threshold, shadows etc)
• I have tried to use GaussianBlur, but that did not help much (maybe I'm using it wrong?)
• I have tried to convert it to grayscale. I'm not sure it made anything better though.
• I have thought about using some sort of morphology, but I don't seem to be able to get it to work. I'm thinking that I have the wrong kernel, and I'm also very uncertain about which kernel to use (or how to create my own)

Here are two images. The one to on top is one that is produced by the program (me holding a tube of shave balm). The one below is one that I have created in paint, and it is like that I want the program to be able to produce on its own. Also note that if I were to have my face in the image instead, the program would produce a much worse result than when I used my hand.

Below, you will see my code. Please come up with suggestions on how to improve my segmentation.

public class Testing {
static {
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
}

static Mat imag = null;
static Mat foreground = null;
static String fileName = "randomTest.png";

public static void main(String[] args) {

    JFrame jFrame = new JFrame("Background subtraction");
    jFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    JLabel videoPanel = new JLabel();
    jFrame.setContentPane(videoPanel);
    jFrame.setSize(640, 480);
    jFrame.setVisible(true);


    double learningRate = 0.001;

    JFrame jForeGround = new JFrame("Foreground shown");
    jForeGround.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    JLabel foreGroundPanel = new JLabel();
    jForeGround.setContentPane(foreGroundPanel);
    jForeGround.setSize(640, 480);
    jForeGround.setVisible(true);

    Mat frame = new Mat();

    VideoCapture camera = new VideoCapture(0);

    Size size = new Size(640, 480);
    BackgroundSubtractorMOG2 mog = Video.createBackgroundSubtractorMOG2(100, 16, false);
    //Mat kernel = Imgproc.getGaussianKernel(3, 1.5);

    while (true) {
        if (camera.read(frame)) {
            Imgproc.resize(frame, frame, size);

            imag = frame.clone();
            foreground = new Mat(frame.size(), CvType.CV_8UC1);
            Imgproc.cvtColor(imag, foreground, Imgproc.COLOR_BGR2GRAY);
            Imgproc.GaussianBlur(foreground, foreground, new Size(7,7), 0);

            mog.apply(imag, foreground, learningRate);
            //Imgproc.dilate(foreground, foreground, kernel);
            ImageIcon image = new ImageIcon(mat2bufferedImage(imag));
            ImageIcon imageForeground = new ImageIcon(mat2bufferedImage(foreground));
            //Imgcodecs.imwrite(fileName, foreground);

            videoPanel.setIcon(image);
            foreGroundPanel.setIcon(imageForeground);
        }
    }
}

private static BufferedImage mat2bufferedImage(Mat image) {
    MatOfByte bytemat = new MatOfByte();
    Imgcodecs.imencode(".jpg", image, bytemat);
    /*
     * Encodes an image into a memory buffer. The first argument is the file
     * extension. The second argument is the image to be written, and the
     * third is the output buffer
     * 
     */
    byte[] bytes = bytemat.toArray();
    /*
     * Creates an array of bytes, and puts bytemat in it
     * 
     */
    InputStream in = new ByteArrayInputStream(bytes);
    BufferedImage img = null;
    try {
        img = ImageIO.read(in);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return img;
}