Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Decoding a h264 (High) stream with OpenCV's ffmpeg on Ubuntu

I am working with a video stream (no audio) from an ip camera on Ubuntu 14.04. Also i am a beginner with Ubuntu and everything on it. Everything was going great with a camera that has these parameters (from FFMPEG):

Input #0, rtsp, from 'rtsp://*private*:8900/live.sdp': 0B f=0/0   
  Metadata:
    title           : RTSP server
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 352x192, 29.97 tbr, 90k tbn, 180k tbc

But then i changed to a newer camera, which has these parameters:

Input #0, rtsp, from 'rtsp://*private*/media/video2':0B f=0/0   
  Metadata:
    title           : VCP IPC Realtime stream
    Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1280x720, 25 fps, 25 tbr, 90k tbn, 50 tbc

My C++ program uses OpenCV3 to process the stream. By default OpenCV uses ffmpeg to decode and display the stream with function VideoCapture.

VideoCapture vc;
vc.open(input_stream);
while ((vc >> frame), !frame.empty()) {
   *do work*
}

With the new camera stream i get errors like these (from ffmpeg):

[h264 @ 0x7c6980] cabac decode of qscale diff failed at 41 38
[h264 @ 0x7c6980] error while decoding MB 41 38, bytestream (3572)
[h264 @ 0x7c6980] left block unavailable for requested intra mode at 0 44
[h264 @ 0x7c6980] error while decoding MB 0 44, bytestream (4933)
[h264 @ 0x7bc2c0] SEI type 25 truncated at 208
[h264 @ 0x7bfaa0] SEI type 25 truncated at 206
[h264 @ 0x7c6980] left block unavailable for requested intra mode at 0 18
[h264 @ 0x7c6980] error while decoding MB 0 18, bytestream (14717)

The image sometimes is glitched, sometimes completely frozen. After a few seconds to a few minutes the stream freezes completely without an error. I tried appending ?tcp to the input stream, but the video still froze without an error after ~10 seconds. However on vlc it plays perfectly. I installed the newest version (3.2.2) of ffmpeg player with

./configure --enable-gpl --enable-libx264

Now playing directly with ffplay (instead of launching from source code with OpenCV function VideoCapture), the stream plays better, doesn't freeze, but sometimes still displays warnings:

[NULL @ 0x7f834c008c00] SEI type 25 size 896 truncated at 320=1/1   
[h264 @ 0x7f834c0d5d20] SEI type 25 size 896 truncated at 319=1/1   
[rtsp @ 0x7f834c0008c0] max delay reached. need to consume packet   
[rtsp @ 0x7f834c0008c0] RTP: missed 1 packets
[h264 @ 0x7f834c094740] concealing 675 DC, 675 AC, 675 MV errors in P frame
[NULL @ 0x7f834c008c00] SEI type 25 size 896 truncated at 320=1/1

Changing the camera hardware is not an option. The camera can be set to encode to h265 or mjpeg. When encoding to mjpeg it can output 5 fps, which is not enough. Decoding to a static video is not an option either, because i need to display real time results about the stream. Maybe i should switch to some other decoder and player? From my research i conclude that i have these options:

  • Somehow get OpenCV to use the ffmpeg player from another directory, where it is compiled with libx264

  • Somehow get OpenCV to use libvlc instead of ffmpeg

One example of switching to vlc is here, but i don't understand it well enough to say if that is what i need. Or maybe i should be parsing the stream in code? I don't rule out that this could be some basic problem due to a lack of dependencies, because, as i said, i'm a beginner with Ubuntu.

  • Use vlc to preprocess the stream, as suggested here.

This is probably slow, which again is bad for real time results. Any suggestions and coments will be appreciated.