java unknown exception when using Mat.put

asked 2016-08-01 17:31:46 -0600

fatman gravatar image

updated 2016-08-02 00:52:14 -0600

I need to allocate rather large matrix using OpenCV 3.1.0. I'm running following code with -Djava.library.path=$MODULE_DIR$\opencv\310\windows\x64\ -Xmx8g arguments:

public class MatTest extends BaseTest {

  static { System.loadLibrary(Core.NATIVE_LIBRARY_NAME);}

  @Test
  public void tooBig() throws IOException {
    float[] data = new float[13320*67294];
    Mat iMatrix = new Mat(13320, 67294, CvType.CV_32FC1);
    iMatrix.put(0, 0, data); //exception here
  }

  @Test
  public void medium() throws IOException {
    float[] data = new float[13918*13240];
    Mat iMatrix = new Mat(13918, 13240, CvType.CV_32FC1);
    iMatrix.put(0, 0, data);
  }
}

The first test works, since the seconds throws (line: iMatrix.put(0, 0, data))

java.lang.Exception: unknown exception

  at org.opencv.core.Mat.nPutF(Native Method)
  at org.opencv.core.Mat.put(Mat.java:953)
  at my.app.MatTest.tooBig(MatTest.java:19)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
  at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
  at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
  at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)

Is it a OpenCV or native library usage limitation? Is there a workaround for such issue?

Edited: attached full code and stacktrace

edit retag flag offensive close merge delete

Comments

13320*67294*4 = 3585424320 , so you're overflowing signed int type used for the count variable here

berak gravatar imageberak ( 2016-08-02 01:14:47 -0600 )edit

maybe you can work around it by putting 2 slices of half the size ?

berak gravatar imageberak ( 2016-08-02 04:14:38 -0600 )edit

I have tried with smaller pieces, same result:

int height = 13320; // = 4 * 3330
int width = 67294;
float[] data = new float[height * width];
int quarter = data.length/4;
Mat iMatrix = new Mat(height, width, CvType.CV_32FC1);
iMatrix.put(0*height/4, 0, Arrays.copyOfRange(data, 0*quarter, 1*quarter));
iMatrix.put(1*height/4, 0, Arrays.copyOfRange(data, 1*quarter, 2*quarter));
iMatrix.put(2*height/4, 0, Arrays.copyOfRange(data, 2*quarter, 3*quarter));
iMatrix.put(3*height/4, 0, Arrays.copyOfRange(data, 3*quarter, 4*quarter));
fatman gravatar imagefatman ( 2016-08-02 13:16:03 -0600 )edit

hmm, sorry for putting you on the wrong track, then (but that's exactly, what i meant)

berak gravatar imageberak ( 2016-08-02 13:18:46 -0600 )edit

I would try to recompile opencv and change this signed int to something bigger. Do you think it would help?

fatman gravatar imagefatman ( 2016-08-02 13:30:48 -0600 )edit

yea, it's obviously some bug in the c++ -> java wrappers. if you want to sink a screwdiver somewhere, this might be a good entry point (count, num, and some others should be unsigned, not signed, imho.)

but that only will kinda double your headroom. even if count is unsigned int, it'll only allow 4294967295 bytes in one go (that's where the previous 'try to ^do it in slices' idea originated.)

berak gravatar imageberak ( 2016-08-02 13:45:44 -0600 )edit

It took me some time to compile opencv but yes, making those variable unsigned int solved the problem. Thanks a lot. I'll create a pull request with this change - hopefully somone can confirm that it won't break anything.

fatman gravatar imagefatman ( 2016-08-03 15:45:50 -0600 )edit
add a comment