why does cv::VideoCapture >> cv::Mat take so long time?

asked 2019-04-05 20:54:10 -0500

opencvddddd gravatar image

On one single denver core of TX2 board, this function itself takes about 10ms. What's the process of VideoCapture>>Mat, why does it take that long time? I know with more cores, it run faster, but now for some reason, this line of code can only run on single core. (denver core is already the fastest core on TX2)

Is there an alternative that is faster than this line but same effect.

int main(int argc, char** argv) { cpu_set_t cpuset; int cpu = 1; CPU_ZERO(&cpuset);
CPU_SET( cpu , &cpuset); sched_setaffinity(0, sizeof(cpuset), &cpuset);

cv::VideoCapture cap;

cap.open("a video file");
cap.set(CV_CAP_PROP_CONVERT_RGB, true);
long long total_num = cap.get(CV_CAP_PROP_FRAME_COUNT);
float fps = cap.get(CV_CAP_PROP_FPS);

cv::Mat ori_frame;

for(int i=0; i<total_num; i++){ 

    float ms;
    auto t_begin = std::chrono::high_resolution_clock::now();

    cap >> ori_frame;

    auto t_end = std::chrono::high_resolution_clock::now();
    ms = std::chrono::duration<float, std::milli>(t_end - t_begin).count();
    printf("Main function: cap time************************************ %f ms\n", ms);

return 0;


edit retag flag offensive close merge delete


VideoCapture can use different backends underneath, most common on Linux are FFmpeg and GStreamer. In some cases GStreamer backend can synchronize decoding with video file FPS.

First, try to determine which backend has been used: export OPENCV_VIDEOIO_DEBUG=1

Second, try to force using FFmpeg backend: cap.open("filename", CAP_FFMPEG)

Third, try to use GStreamer with HW decoding and synchronization turned off: cap.open("filesrc location=<filename.mp4> ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! appsink sync=false", CAP_GSTREAMER);(I didn't check it though, read https://developer.download.nvidia.com... for more details)

mshabunin gravatar imagemshabunin ( 2019-04-09 07:37:15 -0500 )edit

Hi Mshabunin, For your method 2, doesn't change speed, I guess the default back end is FFMPEG. For your method 3, I think Gstreamer capture data from camera not video file, I used above code, error message is Unable to open video.

opencvddddd gravatar imageopencvddddd ( 2019-04-09 20:16:18 -0500 )edit