Reading captured video data using OpenCV in real-time

asked 2019-03-03 10:54:31 -0600

means-to-meaning gravatar image

updated 2019-03-03 12:27:40 -0600

Is there a good way to read frames from a video file in C++ and process them, such that regardless of the frame processing time we always process the latest available frame in real-time and drop all frames in between?

Using this approach the Total processing time = Recorded video length + Time to process the last frame, regardless of the algorithm processing time. OpenCV might already support this.

If you capture data using your camera at 30 or 60 fps and you would like to run an algorithm on it using OpenCV, you can load it and read it frame by frame using the VideoCapture() class. You will process every frame and the Total processing time = Avg. processing time of a frame * Number of frames.

This approach is not suitable to evaluate real-time performance of algorithms. It is particularly problematic for algorithms that require longer than 1/fps to process a frame since they would have dropped some of the frames in a real-time application, which can dramatically alter their accuracy.

edit retag flag offensive close merge delete

Comments

read frames from a video file

that's not a realtime task, in the 1st place (and somehow, a "bogus" problem)

berak gravatar imageberak ( 2019-03-05 03:14:16 -0600 )edit
1

Perhaps I did not ask the question right then! :) When you run a cv algorithm against a streaming camera and you output the results as fast as possible (meaning you are trying to achieve a decent fps > 1), that is 'real-time' computer vision for me. If you feed in frames offline in a loop into this algorithm using OpenCV it does not re-create the above conditions well since every frame is processed, i.e. there are no dropped frames. Perhaps you could help me ask the question better?

means-to-meaning gravatar imagemeans-to-meaning ( 2019-03-09 02:57:34 -0600 )edit

opencv's VideoCapture might be a poor choice for time-critical applications, maybe you need to fall back to other tools, native SDK's, etc. but you can try 2 things:

  • seek to a frame in the future: cap.set(frameno). unfortunately, for a few frames diff, it might be more costly seeking the prev. keyframe, and decoding all frames to the desired pos, than just playing them one by one
  • skipping the decoding: cap.read() is a combination of cap.grab() (get it) and cap.retrieve() (decode it). if you fall behind, you might try to skip the retrieve() for a few times.
berak gravatar imageberak ( 2019-03-09 03:09:58 -0600 )edit
add a comment