opencv java multiple thread memory leak
I'm using OpenCV's(3.4.1) JNI wrapper as port for to use OpenCV in Java, and i need handle a lots jpeg images.
It's fine and no memory leak if handle in single thread. But huge memory leak when using multiple threads.
Can anyone help me fix this?
My code:
int nThreads = 128;
ExecutorService pool = Executors.newFixedThreadPool(nThreads);
while (true) {
CountDownLatch ready = new CountDownLatch(nThreads);
CountDownLatch start = new CountDownLatch(1);
CountDownLatch end = new CountDownLatch(nThreads);
for (int i = 0; i < nThreads; i++) {
int index = i;
pool.execute(() -> {
ready.countDown();
try {
start.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
Mat mat = Imgcodecs.imread(files[index].getAbsolutePath(), Imgcodecs.IMREAD_COLOR);
mat.free(); // same as release() and n_delete()
end.countDown();
});
}
ready.await();
start.countDown();
end.await();
System.out.println("Sleep");
Thread.sleep(5000);
}
mat.free()-- does it exist, even?same for n_delete() -- a chimera.
also, finalize() only moves it to gc (but won't release c++ allocated memory)
I add free method and remove final modifier nativeObj filed in Mat class for release quickly, the code is
opencv is still a c++ library NOT a java one.
you have to be extremely careful, when trying to apply usual java paradigms, like assuming, any of it would be threadsafe (hint: it is only where you find an explicit notice in the c++ docs about being so !)
"I add free method " -- and you modified the protocol.
noone knows, what you're doing, and in the end, you get, what you deserve..
rule #1 : assume it's NOT thread-safe.
(though just using imread() might even be)
then: reading images from disk is mainly an IO problem, you can't gain anything with threading.
Thanks for you hints. It's just a little change and doesn't break the protocol of Mat class. The same issue when using original Mat class and even worse because you can't control the finalize method, it's called by jvm gc. I just curiousness why it cause huge memory leak in java multiple thread. May be i should move the multiple threads handle to c++ code?
no. the brutal answer is: when using opencv, avoid multi-threading at all.
(there are tons of "under-the-hood" optimization here, like static opencl buffers, that assume, you're NOT doing this.)
yes, that's been critisized before.
and ahh, now i see, where you're heading ..
I checked all related issues from github, but still can't figure out this issue.