Low FPS when using CascadeClassifier in Android
I'm new to OpenCV and I've loaded the sample face detection app onto my Android device (Samsung Galaxy Nexus) and it works brilliantly, with a FPS around 20-25 FPS (depending on lighting conditions).
However, when I go to make my own code (which is highly based upon the sample code), the frame rate drops considerably, and now I'm only getting around 1-2 FPS.
I'm following the basic skeleton of the sample application, and the important work is done in the onCameraFrame
method, which is as follows:
@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
// this method recieves the input frame from the camera BEFORE it is drawn on the
// screen of the device. The face detection happens here.
rgba = inputFrame.rgba();
gray = inputFrame.gray();
MatOfRect faces = new MatOfRect(); //dense for each face
if (javaDetector != null) {
javaDetector.detectMultiScale(gray, faces, 1.1, 2, 2, new Size(0, 0.2f), new Size());
}
Rect[] arrayFaces = faces.toArray();
for (int i = 0; i < arrayFaces.length; i++) {
Core.rectangle(rgba, arrayFaces[i].tl(), arrayFaces[i].br(), rectangleColor, rectangleColor, 3);
}
return rgba;
}
The code that I have to load the library (in the BaseLoaderCallback
method) is here:
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
@Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
File classifierDir = getDir("cascade", Context.MODE_PRIVATE);
try {
InputStream is = getResources().openRawResource(R.raw.haarcascade_frontalface_alt);
File classifierFile = new File(classifierDir, "haarcascade_frontalface_alt.xml");
FileOutputStream os = new FileOutputStream(classifierFile);
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = is.read(buffer)) != -1) {
os.write(buffer, 0, bytesRead);
}
is.close();
os.close();
javaDetector = new CascadeClassifier(classifierFile.getAbsolutePath());
if (javaDetector.empty()) {
throw new FileNotFoundException();
}
classifierDir.delete();
} catch (FileNotFoundException ex) {
Log.e(TAG, "Unable to load facial definitions");
} catch (IOException iex) {
Log.e(TAG, "Unable to open facial definitions file");
}
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
Which is essentially the same as the sample application. Is there a reason why that my version is running so slowly?
iirc, the sample app is using a lbp cascade, not a haar one (they're like 3 times faster!)
also, the 1st Size() arg in detectMultiScale should be the maximum Size checked for a face. your one looks a bit broken there
yeah, i tried it too, and the performance is still dramatically lower (now averaging around 3-5 FPS using the lbpcacade, instead of 1-2 when using the haar classifier)
could you shed some light on your native jni code ? why even use jni for this ? what are you doing different from the builtin c++ <--> java wrappers there ?
That's the thing, I don't have any native jni code as far as I'm aware. I'm not trying to do anything different either, I just wanted to see if I couldn't replicate the sample app on my own.
oh, you had some (seemingly random now) jni/native related tags there (before i killed them ;), so i probably misread you. is 'javaDetector' a plain CascadeClassifier ?
yeah, it's just a plain CascadeClassifier that loads the lbp/haar cascade from the file.
again, adjust your Size(0, 0.2f) argument in detectMultiScale (it will get clamped to Size(0,0), meaning - check the whole image !).
the tutorial code restricts it to Size(mAbsoluteFaceSize, mAbsoluteFaceSize) there, search time goes O^2 with the area.
Thanks berak, that turned out to be my issue!
ahh, good to know ;)