Ask Your Question

OpenCV - Streaming H264 over RTSP using FFMPEG in version 3.4

asked 2018-03-16 08:50:54 -0600

skr_robo gravatar image

updated 2018-03-16 14:43:37 -0600

I am trying to capture an RTSP stream from a VIRB 360 camera, into OpenCV. The video is H264 and according to one of the comments here, OpenCV 3.4 should be able to handle it. Here is the code:

#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/videoio.hpp>

int main()
    cv::VideoCapture cap("rtsp://", cv::CAP_FFMPEG);

        std::cout << "Input error\n";
        return -1;

    cv::namedWindow("Video Feed", cv::WINDOW_AUTOSIZE);

    cv::Mat frame;
        //std::cout << "Format: " << cap.get(CV_CAP_PROP_FORMAT) << "\n";
        cap >> frame;
        cv::imshow("Video Feed", frame);    
        if (cv::waitKey(10) == 27)
    return 0;

I have compiled OpenCV with ffmpeg and gstreamer capabilities. When I run the following Gstreamer command, I am able to stream it, but with a delay of 3 seconds (not acceptable):

gst-launch-1.0 playbin uri=rtsp://\&liveStreamActive=1

On the other hand, I get a 0.5 second delay using ffplay/ffmpeg command (acceptable):

ffplay -fflags nobuffer -rtsp_transport udp rtsp://\&liveStreamActive=1


ffplay -probesize 32 -sync ext rtsp://\&liveStreamActive=1

In the OpenCV code written above, using cv::CAP_FFMPEG flag in the line:

cv::VideoCapture cap("rtsp://", cv::CAP_FFMPEG);

gives the error:

[rtsp @ 0x2312040] The profile-level-id field size is invalid (65)
[rtsp @ 0x2312040] method SETUP failed: 461 Unsupported transport
Input error

If I use cv::CAP_GSTREAMER, it throws no error, but nothing happens. I believe that the problem is that OpenCV is not able to handle UDP transport layer. What are the possible solutions? Kindly provide suggestions.

Edit 1:

I was able to get capture the stream by following this. I made the following changes: instead of cv::VideoCapture cap("rtsp://", cv::CAP_FFMPEG); the code now has:

#if WIN32
        _putenv_s("OPENCV_FFMPEG_CAPTURE_OPTIONS", "rtsp_transport;udp");
        setenv("OPENCV_FFMPEG_CAPTURE_OPTIONS", "rtsp_transport;udp", 1);
    auto cap = cv::VideoCapture("rtsp://", cv::CAP_FFMPEG);

    #if WIN32
        _putenv_s("OPENCV_FFMPEG_CAPTURE_OPTIONS", "");

However, it throws the following errors:

[rtsp @ 0x2090580] The profile-level-id field size is invalid (65)
[rtsp @ 0x2090580] Error parsing AU headers
[h264 @ 0x208d240] error while decoding MB 69 40, bytestream -7
[rtsp @ 0x2090580] Error parsing AU headers
[rtsp @ 0x2090580] Error parsing AU headers
[h264 @ 0x2316700] left block unavailable for requested intra4x4 mode -1
[h264 @ 0x2316700] error while decoding MB 0 16, bytestream 112500
[rtsp @ 0x2090580] Error parsing AU headers

which means the video is sometimes glitchy and looks like: image description

I believe it has something to do with:

setenv("OPENCV_FFMPEG_CAPTURE_OPTIONS", "rtsp_transport;udp", 1);

The third argument 1 was not part of the original solution in the link and was added by me to avoid the error of too few arguments for setenv.

I would appreciate any suggestions ... (more)

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2018-03-16 09:55:14 -0600

BAHRAMUDIN ADIL gravatar image

Streaming videos through RTSP by using FFMPEG is not possible yet with OpenCV. So if you want to stream video on the network, at now the only option available is GStreamer, which needs OpenCV to be built with GStreamer, be default OpenCV does not have GStreamer. By using FFMPEG you are only able to read RTSP stream from the network.

edit flag offensive delete link more


I tried cv::CAP_GSTREAMER option as well. It doesn't display anything.

skr_robo gravatar imageskr_robo ( 2018-03-16 10:33:51 -0600 )edit

@skr_robo using rtsp to connect still having H264 error do you have any idea to solve this?

  1. error while decoding MB 14 16
  2. P sub_mb_type 16 out of range at 62 25
  3. left block unavailable for requested intra4x4 mode -1
  4. cbp too large <3199971767> at 29 25
  5. mb_type 84 in P slice too large at 11 45
  6. out of range intra chroma pred mode
  7. negative number of zero coeffs at 14 35
whdt gravatar imagewhdt ( 2018-08-01 03:22:12 -0600 )edit

@whdt No. sorry.

skr_robo gravatar imageskr_robo ( 2018-08-14 17:12:41 -0600 )edit

You mean that i can not read and display rtsp stream using FFMPEG in opencv?.

BR, Kishan Patel.

kishan patel gravatar imagekishan patel ( 2018-08-21 08:35:15 -0600 )edit

By using FFMPEG you only able to read RTSP stream from the network, but you cannot write the frames to the network, if you want to write the frames over the network by RTMP, you can use GStreamer, but you first build the OpenCV with GStreamer support.

BAHRAMUDIN ADIL gravatar imageBAHRAMUDIN ADIL ( 2018-10-11 10:46:27 -0600 )edit

It means we can open device using FFMPEG and detect face. Right?

kishan patel gravatar imagekishan patel ( 2018-10-12 01:47:10 -0600 )edit

Yes, it is right.

BAHRAMUDIN ADIL gravatar imageBAHRAMUDIN ADIL ( 2018-10-25 11:43:31 -0600 )edit

@kishan patel do you face the problem i list on top?

whdt gravatar imagewhdt ( 2018-10-29 22:04:10 -0600 )edit

Actually i just want to confirm before use FFMPEG.

kishan patel gravatar imagekishan patel ( 2018-10-30 08:11:46 -0600 )edit

Question Tools

1 follower


Asked: 2018-03-16 08:50:54 -0600

Seen: 38,084 times

Last updated: Mar 16 '18