Hi guys,
My research team and i are working on a JRuby multithreaded application, that calls in an on-demand way a simple java class for face-recognition.
the JVM is always Oracle jdk 1.8.0_121 the JRuby version is always jruby 9.1.7.0 about openCV, we have tested the 3.2.0 and 3.1.0 versions, building it with java bindings in the following way:
cmake -DMAKE_BUILD_TYPE=DEBUG -DBUILD_SHARED_LIBS=OFF
putting opencv_320.jar in /usr/share/java, and libopencv_java3**.so in /usr/local/lib .
The java code is the following ( we use two face-detectors ):
public class FaceRecognition {
static{
System.load("/usr/local/lib/libopencv_java310.so");
}
public static String doFaceRec(byte[] img_stream) {
Mat frame = Imgcodecs.imdecode(new MatOfByte(img_stream), Imgcodecs.IMREAD_UNCHANGED);
CascadeClassifier faceDetector1 = new CascadeClassifier("haarcascade_profileface.xml");
faceDetector1.load("haarcascade_profileface.xml");
CascadeClassifier faceDetector2 = new CascadeClassifier("haarcascade_frontalface_alt.xml");
faceDetector2.load("haarcascade_frontalface_alt.xml");
if (faceDetector1.empty() || faceDetector2.empty()) {
frame.release();
System.out.println("faceDetector is empty");
return "0";
}
MatOfRect faceDetections = new MatOfRect();
MatOfRect faceDetections2 = new MatOfRect();
faceDetector1.detectMultiScale(frame, faceDetections, 1.3, 3, 0, new Size(), new Size());
faceDetector2.detectMultiScale(frame, faceDetections2, 1.1, 3, 0, new Size(), new Size());
int found = faceDetections.toArray().length + faceDetections2.toArray().length;
faceDetections.release();
faceDetections2.release();
frame.release();
return ""+found;
}
}
The application generates different threads that call the doFaceRec(..) method provided by FaceRecognition class; through a concurrent-lock mechanism we are sure that only one thread per time calls the doFaceRec(..) method.
We tested the application on different machines with same conditions and/or parameters, with no errors:
- Raspberry Pi 3 (quad-core, 1GB of RAM), Raspbian 8.0 / linux 4.4.38
- Sony vaio (dual-core, 12GB of RAM) Ubuntu 16.04 / linux 4.4.0
- Dell XPS 13 9360 (dual-core, 16GB of RAM), Ubuntu 16.04 / linux 4.8.4
Unfortunately, we had some errors testing the code in other machines:
- Lenovo with i7 quad-core processor, 16GB of RAM, the only machine with openjdk 8
- Amazon m4.4xlarge EC2 instance (16 vCPU, 64gb of RAM) Ubuntu 16.04 / linux 4.4.0
Without any limitation on the number of used threads, and using both opencv 3.2.0 and opencv 3.1.0 , we have two different errors ( we show only the errors on opencv3.2.0 because those with opencv3.1.0 are the same ):
Error 1:
[thread 140705678751488 also had an error][thread 140706560714496 also had an error][thread 140705812969216 also had an error][thread 140705804576512 also had an error][thread 140705787791104 also had an error]#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007ff8dcb7bdcb, pid=3047, tid=0x00007ff89effd700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
C [libopencv_java310.so+0xae3dcb]Aborted (core dumped)
Error 2:
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
java: /dev/opencv/opencv-3.2.0/modules/objdetect/src/cascadedetect.cpp:922: int cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point, int, double&): Assertion `!oldCascade && (data.featureType == FeatureEvaluator::HAAR || data.featureType == FeatureEvaluator::LBP || data.featureType == FeatureEvaluator::HOG)' failed.
Aborted (core dumped)
After, we have tried to limit the number of used threads adding in the Java code the command Core.setNumThreads() provided by openCV:
…
public class FaceRecognition {
static{
System.load("/usr/local/lib/libopencv_java310.so");
Core.setNumThreads(N);
}
public static String doFaceRec(byte[] img_stream) {
Mat frame = Imgcodecs.imdecode(new MatOfByte(img_stream), Imgcodecs.IMREAD_UNCHANGED);
...
and trying the application setting it to 4 and 8 threads, but in both cases (and with both 3.2.0 and 3.1.0 versions) we had the following error:
[thread 139779937117952 also had an error]#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f219dc7ddcb, pid=4028, tid=0x00007f210dffb700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libopencv_java320.so+0xae3dcb]W, cv::HaarEvaluator::OptFeature::calc(int const*) const+0xf
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/ubuntu/spf/src/ruby/bin/hs_err_pid4028.log
#
Aborted (core dumped)
Other notes:
- same errors also executing ulimit -c unlimited before launch the JVM
- same errors also building opencv with TBB library flag (trying to parallelize the detectMultiScale() method )
- we also tried Core.setNumThreads(1) and we don’t have any problem, but we want to take advantage the whole machine power
Conclusions:
We think the problem is into the detectMultiScale() function, because is substantially the only executed operation; we don’t understand the problem because it appears only in some machines, independently from machine’s power. Any help is very appreciated, thank you so much!