Ask Your Question

writing file from Gstreamer pipleine in a VideoCapture on the Tx2

asked 2019-04-05 01:51:19 -0500

yuggieg gravatar image

I'm reading a camera in a GStreamer pipeline on an NVidia TX2. I want the output to be h264 encoded and written to a file. Next to that, I want to have the frames available in OpenCV for further processing.

To do this I made the following pipeline that works using gst-launch:

gst-launch-1.0 nvcamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1280, 
format=(string)I420, framerate=(fraction)30/1' ! tee name=t \
t. ! queue! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! qtmux ! \
filesink \
t. ! queue! nvvidconv ! video/x-raw ! xvimagesink -e

This all works and there are no issues with this pipeline. It shows the frames on screen (this part will be replaced by an appsink in OpenCV) and a moviefile is created that can be played.

Please note the "-e" at the end of of the command. To stop recording I press ctrl-C. If I leave the "-e" out, I get a corrupt moviefile that is not playable. This also makes sense when checking the man-page for gst-launch-1.0, since it mentions specifically this case.

No when I translate this to OpenCV in Python I get this:

import cv2

pipeline = ("nvcamerasrc sensor-id=0 ! "
        "video/x-raw(memory:NVMM), width=(int)640, height=(int)360, format=(string)I420, framerate=(fraction)30/1 ! tee name=t "
        "t. ! queue! omxh264enc ! video/x-h264, stream-format=(string)byte-stream ! "
        "h264parse ! qtmux ! "
        "filesink "
        "t. ! queue! nvvidconv ! video/x-raw, format=(string)BGRx ! "
        "videoconvert ! video/x-raw, format=(string)BGR ! appsink")

vc1 = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
    _, vc1_f =

    cv2.imshow('vc1', vc1_f)
    if not cv2.waitKey(1) == -1:


This also runs fine without any errors, the only thing that happens is that I end up with a corrupt movie file on exit. Exactly the same as when leaving out the "-e" option in the gst-launch version.

In all of this I realise I'm misusing the VideoCapture a bit, because that normally isn't used to write files, but only to get frames into OpenCV through the appsink element. Here, however, it is also used to write a file directly by using the gstreamer pipeline.

I can of course create a separate pipeline:

"appsrc ! videoconvert ! omxh264enc ! h264parse ! qtmux ! filesink "

and feed back the frames I get in OpenCV into that by using a VideoWriter (I tested that and it works properly) but that creates some overhead, so I would prefer to not have to do this and keep it clean and stick to my one VideoCapture pipeline.

So basically my question is: is it possible what I want to do here? Or am I just abusing VideoCapture too much? I.e. is there a Python OpenCV equivalent to the gst-launch "-e" option for a VideoCapture, because VideoCapture::release() doesn't do the job.


edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2019-04-09 05:29:45 -0500

mshabunin gravatar image

VideoCapture does not have this feature:

But VideoWriter has:

You can port some part of the code from VideoWriter to VideoCapture and add a property to control it.

edit flag offensive delete link more


Yeah, I tried that actually. Ran into some issues, mostly because my knowledge of gstreamer is somewhat limited. Thing was that the VideoCapture doesn't have an appsrc so I couldn't inject the EOS into that, as it's done in the VideoWriter. I tried using the solution that gst-launch uses:

But that just caused my code to hang after sending the EOS. That's when I stopped :)

Anyway, thanks for thinking with me!

yuggieg gravatar imageyuggieg ( 2019-04-09 06:02:50 -0500 )edit

answered 2019-04-09 03:27:35 -0500

yuggieg gravatar image

After the comment, I decided to only save the H264 stream and leave the muxing out of the pipeline and do that afterwards separately using ffmpeg. The H264 stream doesn't care about the unclean shutdown. Works well so far.

edit flag offensive delete link more


@yuggieg. My apologized. I posted wrong answer. My co-worker had it. I will post it later.

supra56 gravatar imagesupra56 ( 2019-04-16 04:30:24 -0500 )edit
Login/Signup to Answer

Question Tools



Asked: 2019-04-05 01:51:19 -0500

Seen: 2,228 times

Last updated: Apr 09 '19