Ask Your Question
0

writing file from Gstreamer pipleine in a VideoCapture on the Tx2

asked 2019-04-05 01:51:19 -0500

yuggieg gravatar image

I'm reading a camera in a GStreamer pipeline on an NVidia TX2. I want the output to be h264 encoded and written to a file. Next to that, I want to have the frames available in OpenCV for further processing.

To do this I made the following pipeline that works using gst-launch:

gst-launch-1.0 nvcamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1280, 
format=(string)I420, framerate=(fraction)30/1' ! tee name=t \
t. ! queue! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! \
h264parse ! qtmux ! \
filesink location=test.mov \
t. ! queue! nvvidconv ! video/x-raw ! xvimagesink -e

This all works and there are no issues with this pipeline. It shows the frames on screen (this part will be replaced by an appsink in OpenCV) and a moviefile is created that can be played.

Please note the "-e" at the end of of the command. To stop recording I press ctrl-C. If I leave the "-e" out, I get a corrupt moviefile that is not playable. This also makes sense when checking the man-page for gst-launch-1.0, since it mentions specifically this case.

No when I translate this to OpenCV in Python I get this:

import cv2

pipeline = ("nvcamerasrc sensor-id=0 ! "
        "video/x-raw(memory:NVMM), width=(int)640, height=(int)360, format=(string)I420, framerate=(fraction)30/1 ! tee name=t "
        "t. ! queue! omxh264enc ! video/x-h264, stream-format=(string)byte-stream ! "
        "h264parse ! qtmux ! "
        "filesink location=test.mov "
        "t. ! queue! nvvidconv ! video/x-raw, format=(string)BGRx ! "
        "videoconvert ! video/x-raw, format=(string)BGR ! appsink")


vc1 = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
while(True):
    _, vc1_f = vc1.read()

    cv2.imshow('vc1', vc1_f)
    if not cv2.waitKey(1) == -1:
        break

vc1.release()
cv2.destroyAllWindows()

This also runs fine without any errors, the only thing that happens is that I end up with a corrupt movie file on exit. Exactly the same as when leaving out the "-e" option in the gst-launch version.

In all of this I realise I'm misusing the VideoCapture a bit, because that normally isn't used to write files, but only to get frames into OpenCV through the appsink element. Here, however, it is also used to write a file directly by using the gstreamer pipeline.

I can of course create a separate pipeline:

"appsrc ! videoconvert ! omxh264enc ! h264parse ! qtmux ! filesink location=test.mov "

and feed back the frames I get in OpenCV into that by using a VideoWriter (I tested that and it works properly) but that creates some overhead, so I would prefer to not have to do this and keep it clean and stick to my one VideoCapture pipeline.

So basically my question is: is it possible what I want to do here? Or am I just abusing VideoCapture too much? I.e. is there a Python OpenCV equivalent to the gst-launch "-e" option for a VideoCapture, because VideoCapture::release() doesn't do the job.

Thanks!

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2019-04-09 05:29:45 -0500

mshabunin gravatar image

VideoCapture does not have this feature: https://github.com/opencv/opencv/blob...

But VideoWriter has: https://github.com/opencv/opencv/blob...

You can port some part of the code from VideoWriter to VideoCapture and add a property to control it.

edit flag offensive delete link more

Comments

Yeah, I tried that actually. Ran into some issues, mostly because my knowledge of gstreamer is somewhat limited. Thing was that the VideoCapture doesn't have an appsrc so I couldn't inject the EOS into that, as it's done in the VideoWriter. I tried using the solution that gst-launch uses: https://github.com/GStreamer/gstreame...

But that just caused my code to hang after sending the EOS. That's when I stopped :)

Anyway, thanks for thinking with me!

yuggieg gravatar imageyuggieg ( 2019-04-09 06:02:50 -0500 )edit
0

answered 2019-04-09 03:27:35 -0500

yuggieg gravatar image

After the comment, I decided to only save the H264 stream and leave the muxing out of the pipeline and do that afterwards separately using ffmpeg. The H264 stream doesn't care about the unclean shutdown. Works well so far.

edit flag offensive delete link more

Comments

@yuggieg. My apologized. I posted wrong answer. My co-worker had it. I will post it later.

supra56 gravatar imagesupra56 ( 2019-04-16 04:30:24 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2019-04-05 01:51:19 -0500

Seen: 2,228 times

Last updated: Apr 09 '19