# writing file from Gstreamer pipleine in a VideoCapture on the Tx2

I'm reading a camera in a GStreamer pipeline on an NVidia TX2. I want the output to be h264 encoded and written to a file. Next to that, I want to have the frames available in OpenCV for further processing.

To do this I made the following pipeline that works using gst-launch:

gst-launch-1.0 nvcamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1280,
format=(string)I420, framerate=(fraction)30/1' ! tee name=t \
t. ! queue! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! qtmux ! \
filesink location=test.mov \
t. ! queue! nvvidconv ! video/x-raw ! xvimagesink -e


This all works and there are no issues with this pipeline. It shows the frames on screen (this part will be replaced by an appsink in OpenCV) and a moviefile is created that can be played.

Please note the "-e" at the end of of the command. To stop recording I press ctrl-C. If I leave the "-e" out, I get a corrupt moviefile that is not playable. This also makes sense when checking the man-page for gst-launch-1.0, since it mentions specifically this case.

No when I translate this to OpenCV in Python I get this:

import cv2

pipeline = ("nvcamerasrc sensor-id=0 ! "
"video/x-raw(memory:NVMM), width=(int)640, height=(int)360, format=(string)I420, framerate=(fraction)30/1 ! tee name=t "
"t. ! queue! omxh264enc ! video/x-h264, stream-format=(string)byte-stream ! "
"h264parse ! qtmux ! "
"filesink location=test.mov "
"t. ! queue! nvvidconv ! video/x-raw, format=(string)BGRx ! "
"videoconvert ! video/x-raw, format=(string)BGR ! appsink")

vc1 = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
while(True):

cv2.imshow('vc1', vc1_f)
if not cv2.waitKey(1) == -1:
break

vc1.release()
cv2.destroyAllWindows()


This also runs fine without any errors, the only thing that happens is that I end up with a corrupt movie file on exit. Exactly the same as when leaving out the "-e" option in the gst-launch version.

In all of this I realise I'm misusing the VideoCapture a bit, because that normally isn't used to write files, but only to get frames into OpenCV through the appsink element. Here, however, it is also used to write a file directly by using the gstreamer pipeline.

I can of course create a separate pipeline:

"appsrc ! videoconvert ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mov "


and feed back the frames I get in OpenCV into that by using a VideoWriter (I tested that and it works properly) but that creates some overhead, so I would prefer to not have to do this and keep it clean and stick to my one VideoCapture pipeline.

So basically my question is: is it possible what I want to do here? Or am I just abusing VideoCapture too much? I.e. is there a Python OpenCV equivalent to the gst-launch "-e" option for a VideoCapture, because VideoCapture::release() doesn't do the job.

Thanks!

edit retag close merge delete

Sort by » oldest newest most voted

VideoCapture does not have this feature: https://github.com/opencv/opencv/blob...

But VideoWriter has: https://github.com/opencv/opencv/blob...

You can port some part of the code from VideoWriter to VideoCapture and add a property to control it.

more

Yeah, I tried that actually. Ran into some issues, mostly because my knowledge of gstreamer is somewhat limited. Thing was that the VideoCapture doesn't have an appsrc so I couldn't inject the EOS into that, as it's done in the VideoWriter. I tried using the solution that gst-launch uses: https://github.com/GStreamer/gstreame...

But that just caused my code to hang after sending the EOS. That's when I stopped :)

Anyway, thanks for thinking with me!

( 2019-04-09 06:02:50 -0500 )edit

After the comment, I decided to only save the H264 stream and leave the muxing out of the pipeline and do that afterwards separately using ffmpeg. The H264 stream doesn't care about the unclean shutdown. Works well so far.

more

@yuggieg. My apologized. I posted wrong answer. My co-worker had it. I will post it later.

( 2019-04-16 04:30:24 -0500 )edit

Official site

GitHub

Wiki

Documentation