Low FPS when using CascadeClassifier in Android

asked 2014-07-10 14:00:46 -0600

updated 2014-07-10 14:30:58 -0600

berak gravatar image

I'm new to OpenCV and I've loaded the sample face detection app onto my Android device (Samsung Galaxy Nexus) and it works brilliantly, with a FPS around 20-25 FPS (depending on lighting conditions).

However, when I go to make my own code (which is highly based upon the sample code), the frame rate drops considerably, and now I'm only getting around 1-2 FPS.

I'm following the basic skeleton of the sample application, and the important work is done in the onCameraFrame method, which is as follows:

@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    // this method recieves the input frame from the camera BEFORE it is drawn on the
    // screen of the device. The face detection happens here.

    rgba = inputFrame.rgba();
    gray = inputFrame.gray();
    MatOfRect faces = new MatOfRect(); //dense for each face
    if (javaDetector != null) {
        javaDetector.detectMultiScale(gray, faces, 1.1, 2, 2, new Size(0, 0.2f), new Size());
    }

    Rect[] arrayFaces = faces.toArray();
    for (int i = 0; i < arrayFaces.length; i++) {
        Core.rectangle(rgba, arrayFaces[i].tl(), arrayFaces[i].br(), rectangleColor, rectangleColor, 3);
    }
    return rgba;
}

The code that I have to load the library (in the BaseLoaderCallback method) is here:

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
    @Override
    public void onManagerConnected(int status) {
        switch (status) {
            case LoaderCallbackInterface.SUCCESS:
            {
                File classifierDir = getDir("cascade", Context.MODE_PRIVATE);
                try {
                    InputStream is = getResources().openRawResource(R.raw.haarcascade_frontalface_alt);
                    File classifierFile = new File(classifierDir, "haarcascade_frontalface_alt.xml");
                    FileOutputStream os = new FileOutputStream(classifierFile);

                    byte[] buffer = new byte[4096];
                    int bytesRead;
                    while ((bytesRead = is.read(buffer)) != -1) {
                        os.write(buffer, 0, bytesRead);
                    }
                    is.close();
                    os.close();

                    javaDetector = new CascadeClassifier(classifierFile.getAbsolutePath());
                    if (javaDetector.empty()) {
                        throw new FileNotFoundException();
                    }
                    classifierDir.delete();
                } catch (FileNotFoundException ex) {
                    Log.e(TAG, "Unable to load facial definitions");
                } catch (IOException iex) {
                    Log.e(TAG, "Unable to open facial definitions file");
                }
                mOpenCvCameraView.enableView();
            } break;
            default:
            {
                super.onManagerConnected(status);
            } break;
        }
    }
};

Which is essentially the same as the sample application. Is there a reason why that my version is running so slowly?

edit retag flag offensive close merge delete

Comments

1

iirc, the sample app is using a lbp cascade, not a haar one (they're like 3 times faster!)

also, the 1st Size() arg in detectMultiScale should be the maximum Size checked for a face. your one looks a bit broken there

berak gravatar imageberak ( 2014-07-10 14:08:36 -0600 )edit

yeah, i tried it too, and the performance is still dramatically lower (now averaging around 3-5 FPS using the lbpcacade, instead of 1-2 when using the haar classifier)

docaholic gravatar imagedocaholic ( 2014-07-10 14:14:10 -0600 )edit
1

could you shed some light on your native jni code ? why even use jni for this ? what are you doing different from the builtin c++ <--> java wrappers there ?

berak gravatar imageberak ( 2014-07-10 14:41:29 -0600 )edit

That's the thing, I don't have any native jni code as far as I'm aware. I'm not trying to do anything different either, I just wanted to see if I couldn't replicate the sample app on my own.

docaholic gravatar imagedocaholic ( 2014-07-10 15:09:55 -0600 )edit

oh, you had some (seemingly random now) jni/native related tags there (before i killed them ;), so i probably misread you. is 'javaDetector' a plain CascadeClassifier ?

berak gravatar imageberak ( 2014-07-10 15:11:18 -0600 )edit

yeah, it's just a plain CascadeClassifier that loads the lbp/haar cascade from the file.

docaholic gravatar imagedocaholic ( 2014-07-10 15:15:10 -0600 )edit
1

again, adjust your Size(0, 0.2f) argument in detectMultiScale (it will get clamped to Size(0,0), meaning - check the whole image !).

the tutorial code restricts it to Size(mAbsoluteFaceSize, mAbsoluteFaceSize) there, search time goes O^2 with the area.

berak gravatar imageberak ( 2014-07-10 15:24:24 -0600 )edit

Thanks berak, that turned out to be my issue!

docaholic gravatar imagedocaholic ( 2014-07-10 15:32:21 -0600 )edit

ahh, good to know ;)

berak gravatar imageberak ( 2014-07-10 15:48:29 -0600 )edit