memory leak in Mat.put(int,int,byte[])? or wrong usage? (Java)

asked 2018-02-07 09:35:31 -0500

kaba gravatar image

G'day!

My application is reading in a lot of BufferedImage, eventually transforming them into OpenCv-Matobjects for processing. Ho to do this is described - e.g. - here: https://stackoverflow.com/questions/1... . (The application is using the BI _and_ the Mat, so the BI are needed.)

There seems to be a problem, though: it seems the application is eating up memory during conversion - native memory, not Java heap. I narrowed down this (suspected) memory leak to setting the Mat contents from a byte[]. Could anyone confirm this is a memory leak, or (preferably) explain to me, that I'm using Mat in an incorrect way here (and hint me to the right path)? I searched the big, wide WWW for a while now and couldn't find anything that seems to match.

Below you'll find a code snippet that showcases my findings. The variant 1 emulates there being different images getting converted to Mat; variant 2 simplifies the example, but also drastically changes it's behaviour. I chose sizes to get my system into using swap, so be sure to adjust the numbers before running the example. The uses of showMessageDialog could also be substituted by e.g. Thread.sleep(...).

Observation: the code, when run, allocates more memory than I'd expect. The first gc() will do nearly nothing. The calls to Mat.release() seem to free up quite a lot of memory (as expected), but a lot stays allocated. Letting go of the object reference to mats doesn't change anything.

Latest at this point I'd expect all memory (byte[] and Mat) to be freed, but it isn't - and the memory still allocated is not within the Java heap, so must be native memory. The memory not getting freed seems to be the memory allocated for the byte[], which can be shown by trying variant 2 instead of variant 1.

I'm using OpenCV 3.2.0 here, more precise: the maven-distributed version of https://github.com/openpnp/opencv . 3.2.0 is not the latest version, but fairly recent. Anyhow: any hints that this problem was already addressed between 3.2.0 and the latest release would also be welcome.

Any ideas?

The code snippet [retyped here, so pls be merciful in case of typos]:

final int dim = 10000;  // vvv
final int count = 100;   // these should amount to ~ 10G mem usage, adjust as needed
// byte[] pixels = new byte[ dim * dim];  // Variant 2
Mat[] mats = new Mat[ count ];
for( int i=0; i<mats.length; i++ ) {
    mats[i] = new Mat( dim, dim, CvType.CV_8UC1 );
    byte[] pixels = new byte[ dim * dim];  // Variant 1
    mats[i].put( 0, 0, pixels );
}
System.out.println("allocated");
JOptionPane.showMessageDialog( null, "click to continue", "Godot", JOptionPane.INFORMATON_MESSAGE );
System.gc();
System.out.println("gc'd");
JOptionPane.showMessageDialog( null, "click to continue", "Godot", JOptionPane.INFORMATON_MESSAGE );
Arrays.asList( mats ).parallelStream().forEach( m -> m.release() );
System.gc();
System.out.println("released mats and gc'd");
JOptionPane.showMessageDialog( null, "click to continue", "Godot", JOptionPane.INFORMATON_MESSAGE ...
(more)
edit retag flag offensive close merge delete

Comments

Today I examined if there's a bug in the OpenCV Java connection layer. But it seems that the Java array is correctly released after usage in the native code; so no bug there, as far as I can see that. (see https://github.com/opencv/opencv/blam... , line 1848)

kaba gravatar imagekaba ( 2018-02-08 06:36:05 -0500 )edit