Java: Memory leak from iterating OpenCV frames

asked 2014-01-11 07:02:51 -0500

Matthias gravatar image

updated 2014-01-11 12:06:42 -0500

I am using the java wrapper of OpenCV. I tried to write an Iterator over frames of a film. My problem is that the iterator is a huge memory leak. It fills a GByte within seconds. Here is a very simplified version of the iterator, which has this leak:

public final class SimpleIt implements Iterator<Mat> {

    private final VideoCapture capture;
    boolean hasNext;

    public SimpleIt(final VideoCapture capture) {
        this.capture = capture;
        hasNext = capture.grab();
    }

    @Override
    public boolean hasNext() {
        return hasNext;
    }

    @Override
    public Mat next() {
        final Mat mat = new Mat();
        capture.retrieve(mat);
        hasNext = capture.grab();
        return mat;
    }
}

When I Iterate over this code using e.g. this loop:

    final VideoCapture vc = new VideoCapture("/path/to/file");
    final SimpleIt it = new SimpleIt(vc);
    while (it.hasNext) {
        it.next();
    }

memory consumption will increase linear. I see that the problem is the first line in the next()-Method. It always creates a new Mat. But speaking of java alone, this Mat will run out of scope as soon as the it.next() statement is over.

I could overcome the problem, by not using a new Mat every time, but overwriting always the same Mat-Object, like this:

    private final VideoCapture capture;
    private final Mat mat = new Mat();
    boolean hasNext;

    @Override
    public Mat next() {
        capture.retrieve(mat);
        hasNext = capture.grab();
        return mat;
    }

But now the last frame which was given by the iterator will always be overwritten by the current frame. Thus, I cannot hold it outside for later use, if I am interested in this single frame. I could copy it, of course, but that would also be expensive.

To illustrate the problem with the last approach, imagin this code using the iterator:

    final VideoCapture vc = new VideoCapture("/path/to/file");
    final SimpleIt it = new SimpleIt(vc);
    int i = 0;
    Mat save = null;
    while (it.hasNext) {
        final Mat next = it.next();
        if (i == 10) {
            save = next;
            Highgui.imwrite("/path/to/10.png", save);
        } else if (i == 30) {
            Highgui.imwrite("/path/to/30.png", save);
        }
        i++;
    }

With the second version of the iterator, 10.png, and 30.png will be different images. But that's obviously not what was intended.

I assume that the problem is that the garbage collector will not destroy the Mat objects, because it does not recognize the memory consumption, since it is not java heap space. Calling mat.release() in the loop will help, but of course in real code this means I will have no garbage collection for my Mat objects.

Anybody has an idea how to do it?

Edit: I expermiented now a while with it, and came to the following solution:

    int count = 0;

    @Override
    public Mat next() {
        final Mat result = mat;
        mat = new Mat();
        capture.retrieve(mat);
        hasNext = capture.grab();
        if (++count % 200 == 0) {
            System.gc();
        }
        return result;

If I call the garbage collector on a regular basis, it will delete all the unreferenced Mat objects, and together with them the RAM allocated from C. Well, that's probably a problem you have to face when mixing ... (more)

edit retag flag offensive close merge delete

Comments

could you explain, why you need that iterator construct at all ?

why not just call capture.read(mat) ; and bail out if it failed(last frame reached)

berak gravatar imageberak ( 2014-01-11 11:00:01 -0500 )edit

First of all, doing it in a loop instead of an iterator won't help too much. I could simply use the second version of the iterator, and just copy the frames away, which I need for later.

And of course, I just posted a minimized example here. The original iterator is way more complex. E.g. it starts a new thread to retreive the pictures simultaneously to processing the old, "interesting" pictures found so far.

Matthias gravatar imageMatthias ( 2014-01-11 11:51:18 -0500 )edit

ah, ok. i see. thanks for explaining.

berak gravatar imageberak ( 2014-01-11 12:02:00 -0500 )edit