Process video frame by frame in OpenCV on Android

asked 2014-06-13 01:52:26 -0500

Vion gravatar image

updated 2014-06-13 02:28:01 -0500

berak gravatar image

Hey all,

my goal is as follows: I have to read in a video that is stored on the sd card, process it frame for frame and then store it in a new file on the SD card again. The problem is that OpenCV4Android does not come with an video encoder/decoder as it does not include ffmpeg. Moreover, using JavaCV for processing the image is no option for me, as the code is already written in native OpenCV and I access them through the JNI. I did a lot of reading here on stackoverflow and the rest of google. But I did not find the solution.

1) JavaCV allows me to read a video frame by frame and also store it frame by frame. However, I am not able to convert the video to plain OpenCV Mat objects which can be processed by usual OpenCV4Android.

2) I read about JCodec as a library for encoding/decoding videos. Would JCodec allow me to fulfill my task? If yes, do you know any examples.

3) Compiling FFMPEG for Android would also be an option. However, I think it is a bit overkill to write FrameGrabber and FrameRecorder my self. I think that there must exist some solution besides the one of JavaCV.

4) Starting with API 18 there are the MediaCodec and the MediaMuxer in Android. Perhaps they can help me?

So lets come to my requirements. So I'm currently targeting Android API 19, so I have every function available which I need. The most important requirement for me is the following:

If I process a video of 10 seconds with 30 FPS, the result should also be a video of 10 seconds with 30 FPS. So I want an exact copy of the video but with some drawing added to each frame by OpenCV. Using OpenCV via Python for example can do this task by using the VideoWriter class and VideoInput class. I need the same functionality on Android.

I am wondering that no one had this problem so far (or I did not find it).

Hopefully, I explained everything.

Best regards, André

edit retag flag offensive close merge delete


Did you get any help on this? I'm in the same boat.

valder gravatar imagevalder ( 2014-08-22 13:58:56 -0500 )edit

@valder, unfortunately, the situation has not changed much over a year.

berak gravatar imageberak ( 2014-08-22 14:06:04 -0500 )edit

@berak, I"m having a hard time extracting frames from a video. Can you show me what you did to do this? All the frames I extract come out all garbled. I'm not sure what is wrong. Any help would be appreciated.

valder gravatar imagevalder ( 2014-08-25 18:41:39 -0500 )edit