OpenCV multi-threading frames on Android
I am doing some work with OpenCV on Android. My code was all on the Java interface and its performance was very bad, so I implemented almost all the image processing with JNI. So on Java I have:
@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame)
{
_imgBgrTemp = inputFrame.rgba();
jniFrame(_imgBgrTemp.nativeObj);
return _imgBgrTemp;
}
So jniFrame() function takes care of all the image processing in C++, and with this there was a good improvement in performance. But it is still around 8fps and my camera can do 30fps without any processing on the image.
Looking more close I saw that even while processing the code, it uses only 25% CPU of my Android, witch is a Zenfone 2 with a quad-core processor.
I am thinking about having it running in 4 threads so I would have o FIFO pool to receive frames, process, and display it in the received order.
I am thinking in use this: Creating a Manager for Multiple Threads
So my questions are:
I am going the right way ?
Do you have any tips ?
What should I consider (As I am working with JNI) ?
I didn't post the jniFrame() here because I don't think it is very relevant as it is a Work in progress code, very big. But it is about recognizing a rubik cube and getting its colors. if you also can give me any tips on that... but I may create another question only about this part later.
An update: I as just searching about using OpenCL, but it seeams even more complicated then multi-threading and I am not sure if the improvement would be good. would it be better then multi-threading ?
You can also subsample your input image if it doesn't affect your vision task.
Hi. Sorry but I didn't really understand what you meant by it.
Instead of working on full resolution image (e.g. 1920x1080), you could also subsample the image to 640x480 and do your processing on this reduced image if this doesn't affect your vision task.
For example, if you want to detect faces, you can do it on 640x480 instead of 1920x1080.
Yes. Reducing the resolution does help a lot, but I am already doing it, I am working with 640x360. But it is some heavy processing and I only have like 6fps and I should have at least 20fps for it to work great