Hello everybody. I'm trying to build a pipeline to decode an h264 stream coming from a rtsp camera. To achieve this I'm using VideoCapture with GStreamer as backend. This is my currently working pipeline:
"rtspsrc location=rtsp://root:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=3840x2160 protocols=GST_RTSP_LOWER_TRANS_TCP ! rtph264depay ! queue ! vaapih264dec ! videoconvert ! appsink"
As the video is 4K I'm exploiting hardware decoding via VA-API. When I use the above pipeline the cpu usage is aroud 80% and I'm only performing the read() function, but when I use this GStreamer command from terminal
gst-launch-1.0 rtspsrc location="rtsp://root:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=3840x2160" protocols=GST_RTSP_LOWER_TRANS_TCP ! rtph264depay ! queue ! vaapih264dec ! vaapisink
the cpu usage is around 20%. From what I red online, my guessing is that the gsteamer's vaapi plugins use a memory space called VAsurface and also they output a video stream in a format which is not BGR but most probably something like NV12 or I420. The 80% cpu usage might be due to the change of memory space needed by OpenCV to perform its operations, and the change of colorspace from NV12 to BGR.
If that is the case, I've seen that GStremer has a plugin called vaapipostproc and one of its properties is format but no matter what I try, VideoCapture doesn't open. My attempt right now is to perform the change of color space direcly in hardware in order to save some cpu. Here are some extracts of the last piecies of pipeline I've tried:
1) "... ! rtph264depay ! queue ! vaapih264dec ! vaapipostproc format=bgr ! videoconvert ! autovideosink"
2) "... ! rtph264depay ! queue ! vaapih264dec ! vaapipostproc ! video/x-raw,format=(string)BGR ! autovideosink"
If from the last one I change vaapih264dec + vaapipostproc whit vaapidecodebin i get this error: could not link vaapipostproc0 to autovideosink0, vaapipostproc0 can't handle caps video/x-raw, format=(string)BGR but the GStreamer docs states that one of the output format can be the one I want.
Do you have any suggestions on how to use the vaapi plugins of gstreamer correctly in order to perform the change in colorspace and save cpu?
Many thanks