Earth Mover's Distance EMD() results in memory access error

asked 2012-07-20 11:36:21 -0600

gerrit
53 ●1 ●2 ●8

updated 2012-07-20 11:41:03 -0600

Hello,

i am trying to compute the Earth Mover's Distance using opencv's function EMD(). For most of my signatures that works fine. But for others i get an memory access error which i do not understand. At first i thought maybe the signature matrices were to big because they have 39015 rows and 3 columns and the other one has 9216 row and 3 cloumns. But that can not be the problem since a matrix with 36481 rows works fine. Until now i have not been able to tell exactly under which conditions i get this memory access error. But i was able to find the line in opencv where the error occures. That happens to be line 385 in OpenCV-2.4.2/modules/imgproc/src/emd.cpp. The error occours in earlier version like opencv 2.1. I call the function with "cv::EMD(signatureGroundTruthMat, signatureDetectionMat, CV_DIST_L2)". I could provide the matrices that cause the error if that would help. My question would be whether that is a bug in opencv that i should report or whether i have done something wrong with the matrices. Out of 1000 matrices i used EMD() on just a hand full produced that error. I would be thankful for any assistance

Gerrit

answered 2012-07-20 12:43:25 -0600

Ilya Lysenkov
1194 ●6 ●15 ●24 http://twitter.com/ily...

Hi,

I cannot run the code now but I guess there is int overflow in the line 343.

Try to change the emd.cpp like this:

334     long long buffer_size;
...
343     buffer_size = static_cast<long long> (size1+1) * (size2+1) * (sizeof( float ) +    /* cost */

Does it solve the problem?

edit flag offensive delete link

Comments

Hello Ilya, thanks for your help, it seems to at least solve the memory access error. I today made the changes and the calculation started without the error. But it takes a lot mor time to calculate. Most of my signatures take about 10 minutes. The one i tested on after three hours still wasn't finished. I'll let it calculate over night and see whether it finishes. Then i will re-check how long the signatures which are bigger than the ones that crashed before take to compute. For now it seems as if there still would be a bigger issue. In any way shouldn't there an exception be thrown? I will see the results tomorrow and will report back. Kind regards Gerrit

gerrit ( 2012-07-23 15:49:22 -0600 )edit

Hello again, it calculated the signatures. It seems it was finished shortly after i checked the last time. The used memory was about 3gb but that seems reasonable since a matrix half the size seems to take around 1,5gb before i applied your patch to increase the buffersize. And the calculation-time of that half-size matrix is 1 hour 18 minutes. So it would seem that the problem just was the buffersize. But i still think that in such a case opencv should throw an exception instead of exiting with an memory access error. So i think that i will report this as a bug as soon as i have the time. Thanks again for your help! Kind regards Gerrit

gerrit ( 2012-07-24 04:00:13 -0600 )edit

Hello, you are welcome :-)

It was definitely a bug so you should report it at http://code.opencv.org to make sure that it will be fixed.

Ilya Lysenkov ( 2012-07-24 15:48:28 -0600 )edit

I have the same issue: with 16k rows I get a write access exception but if I reduce the size, just to confirm this is the source of the problem, I get no error. There indeed seems to be a bug of maximum buffer size being too small for some cases. I will try to make the changes you suggested and keep you posted if it fixes the problem or not.

pierre_w ( 2018-04-27 04:13:28 -0600 )edit

add a comment

Earth Mover's Distance EMD() results in memory access error

1 answer

Comments

Links

Question Tools

Stats

Related questions

Earth Mover's Distance EMD() results in memory access error edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

Earth Mover's Distance EMD() results in memory access error