Improve background segmentation through BackgroundSubtractMOG2

asked 2016-09-22 08:30:03 -0600

SniFFzoR gravatar image


I have created a small program that is supposed to detect changes in the foreground over a background. It works, but not as well as I would like it to. Here is what I have done so far, to try to improve it on my own.

  • Tweaked parameters (such as history length, learningRate, threshold, shadows etc)
  • I have tried to use GaussianBlur, but that did not help much (maybe I'm using it wrong?)
  • I have tried to convert it to grayscale. I'm not sure it made anything better though.
  • I have thought about using some sort of morphology, but I don't seem to be able to get it to work. I'm thinking that I have the wrong kernel, and I'm also very uncertain about which kernel to use (or how to create my own)

Here are two images. The one to on top is one that is produced by the program (me holding a tube of shave balm). The one below is one that I have created in paint, and it is like that I want the program to be able to produce on its own. Also note that if I were to have my face in the image instead, the program would produce a much worse result than when I used my hand.

image description image description

Below, you will see my code. Please come up with suggestions on how to improve my segmentation.

public class Testing {
static {

static Mat imag = null;
static Mat foreground = null;
static String fileName = "randomTest.png";

public static void main(String[] args) {

    JFrame jFrame = new JFrame("Background subtraction");
    JLabel videoPanel = new JLabel();
    jFrame.setSize(640, 480);

    double learningRate = 0.001;

    JFrame jForeGround = new JFrame("Foreground shown");
    JLabel foreGroundPanel = new JLabel();
    jForeGround.setSize(640, 480);

    Mat frame = new Mat();

    VideoCapture camera = new VideoCapture(0);

    Size size = new Size(640, 480);
    BackgroundSubtractorMOG2 mog = Video.createBackgroundSubtractorMOG2(100, 16, false);
    //Mat kernel = Imgproc.getGaussianKernel(3, 1.5);

    while (true) {
        if ( {
            Imgproc.resize(frame, frame, size);

            imag = frame.clone();
            foreground = new Mat(frame.size(), CvType.CV_8UC1);
            Imgproc.cvtColor(imag, foreground, Imgproc.COLOR_BGR2GRAY);
            Imgproc.GaussianBlur(foreground, foreground, new Size(7,7), 0);

            mog.apply(imag, foreground, learningRate);
            //Imgproc.dilate(foreground, foreground, kernel);
            ImageIcon image = new ImageIcon(mat2bufferedImage(imag));
            ImageIcon imageForeground = new ImageIcon(mat2bufferedImage(foreground));
            //Imgcodecs.imwrite(fileName, foreground);


private static BufferedImage mat2bufferedImage(Mat image) {
    MatOfByte bytemat = new MatOfByte();
    Imgcodecs.imencode(".jpg", image, bytemat);
     * Encodes an image into a memory buffer. The first argument is the file
     * extension. The second argument is the image to be written, and the
     * third is the output buffer
    byte[] bytes = bytemat.toArray();
     * Creates an array of bytes, and puts bytemat in it
    InputStream in = new ByteArrayInputStream(bytes);
    BufferedImage img = null;
    try {
        img =;
    } catch (IOException e) {
    return img;
edit retag flag offensive close merge delete



About morphological operator you should use a closing operator and may be iterate to fill hole

LBerger gravatar imageLBerger ( 2016-09-22 14:41:07 -0600 )edit

Okay, but how do I call for that particular method, and which kernel should I use?

SniFFzoR gravatar imageSniFFzoR ( 2016-09-22 14:55:05 -0600 )edit

I don't know java in C++ something like this :

morphologyEx(src, dst, MORPH_CLOSE,getStructuringElement(MORPH_RECT ,Size(7,7)),Point(-1,-1),5);

kernel is size 7X7 and 5 iterations. I think you will fill black hole of size 35


LBerger gravatar imageLBerger ( 2016-09-22 15:16:49 -0600 )edit

Okay, I can get that to work with this line of Java code (for people that are looking for the syntax): Imgproc.morphologyEx(foreground, foreground, Imgproc.MORPH_CLOSE, Imgproc.getStructuringElement(Imgproc.CV_SHAPE_RECT, new Size(7,7)), new Point(-1, -1), 5);

I just want to know; how did you know what parameters to pass to the getStructuringElement method? Also, how did you know what parameters to pass to the Point class?

SniFFzoR gravatar imageSniFFzoR ( 2016-09-23 08:20:44 -0600 )edit
LBerger gravatar imageLBerger ( 2016-09-23 13:04:10 -0600 )edit