Ask Your Question
5

Camera Preview slow on Android OpenCV +possible bottleneck found

asked 2013-11-08 14:22:52 -0500

scorpeeon gravatar image

updated 2013-12-05 11:14:39 -0500

So I noticed since a long time and wondered ever since why (on multiple devices) using JavaCameraView (not doing any processing just displaying preview images from camera), the smoothness of the camera preview is generally lacking compared to "regular" Android apps using the camera, or even Android OpenCV apps where instead of using JavaCameraView the camera image is manually converted to formats supported by OpenCV. (like this one: https://github.com/ikkiChung/MyRealTimeImageProcessing)

As I understand the "recommended" way to use the camera in OpenCV apps on Android (according to the tutorials and sample apps) is to use a CameraBridgeViewBase implementation (JavaCameraView or NativeCameraView), implement the CvCameraViewListener2 interface and override the onCameraFrame() method where you can process the camera frame.

I found a possible bottleneck which could be the the deliverAndDrawFrame method of the CameraBridgeViewBase.

The problem might be that in the implementation of JavaCameraView the CameraWorker (which would call deliverAndDrawFrame which calls onCameraFrame()) is running on a separate thread and while they're synchronized, the CameraWorker is not always receiving the frames from onPreviewFrame, every now and then it's skipping frames.

I verified this by making a class which extended the JavaCameraView and overrode the onPreviewFrame and logged the calls of this method:

@Override
public void onPreviewFrame(byte[] frame, Camera arg1) {
    Log.i(TAG,"onPreviewFrame called");
    super.onPreviewFrame(frame, arg1);
}

I also logged the onCameraFrame occurrences, calling Log.i(TAG,"onCameraFrame called"); in it.

If working fine this should output alternately the two logs like this: onPreviewFrame called onCameraFrame called onPreviewFrame called onCameraFrame called etc.

While this is true in my test in about 90% of the cases, I can find occurrences where "onPreviewFrame called" appears multiple times before another "onCameraFrame called" message like this: onPreviewFrame called onCameraFrame called onPreviewFrame called onPreviewFrame called onCameraFrame called

This means there are frames from the camera that NEVER make it to the onCameraFrame method. I suspect this frame skipping probably indicates that the deliverAndDrawFrame method's execution time sometimes exceeds the time between frames from the camera, which I think is really high considering it shouldn't do any hard processing work. The smoothness of other applications (not using JavaCamereView) suggests that this could be done more efficiently. Do you think there's any easy way to make it faster? Doing real-time processing is really hard if receiving frames from camera alone is already not real-time.

I only tested JavaCameraView, but since I think NativeCameraView (which also doesn't appear smooth enough) uses the same deliverAndDrawFrame method it's probably also affected.

Any help/insight is appreciated.

edit retag flag offensive close merge delete

Comments

I suspected the same thing but you have gone far to verify. That very nice. Is there any way to fast it up?

Pervez Alam gravatar imagePervez Alam ( 2014-08-20 02:43:24 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-08-26 09:07:11 -0500

I' ve faced this problem either. I' ve tried to decrease the camera resolution with mOpenCvCameraView.setMaxFrameSize(1280, 720);

However it is still slow. I' ve been trying to work parallel with Threads, but it is still 3.5FPS.

public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    //System.gc();
    carrierMat = inputFrame.gray();
    Thread thread = new Thread(new MultThread(carrierMat, this));
    thread.start();
    try {
        thread.join();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    return carrierMat;
}

My MultThread class is just like this

public class MultThread implements Runnable {
private Mat source;
private Context context;

public MultThread(Mat source, Context context) {
    this.source = source;
    this.context = context;
}

@Override
public void run() {
    //output = General.Threshold(source);
    int x = General.MSERP(source);
    Log.i("MtMTxtDtc:Main","x: " + x);
    if (x > 10){
        ((Vibrator) context.getSystemService(Context.VIBRATOR_SERVICE)).vibrate(500);

    }

}
}
edit flag offensive delete link more

Comments

what is General.MSERP(source); ?

Adams30 gravatar imageAdams30 ( 2016-03-10 15:27:51 -0500 )edit

what is General.MSERP(source); ?

it is raised an error.

Akhil Patel gravatar imageAkhil Patel ( 2018-08-30 04:50:50 -0500 )edit
Login/Signup to Answer

Question Tools

Stats

Asked: 2013-11-08 14:22:52 -0500

Seen: 5,348 times

Last updated: Aug 26 '15