Gstreamer Pipeline to NV12toBGR using GAPI

asked 2020-10-23 18:19:37 -0500

warpstar22 gravatar image

updated 2020-10-26 20:23:26 -0500

I am using a gstreamer pipeline with OpenCV VideoCapture on Jetson Nano. The source is captured in NV12 and I have to convert to BGR in the gstreamer pipeline. In order to optimize it, I thought I would use Graph-API. My understanding is that I would leave the pipeline in the NV12 format and then I would do something like the following for GAPI:

cv::GMat in;
cv::GMat out = cv::gapi::NV12toBGR(in);
cv::GComputation ac(in, out);
//videocapture stuff
ac.apply(input, output);

But as it turns out, NV12toBGR has two parameters for y and uv respectively. How are you supposed to use this function alongside a gstreamer pipeline that would capture in NV12?

My gstreamer pipeline for VdieoCapture typically looks like this:

nvarugscamerasrc sensor_id=0 ! video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12 ! nvvidconv ! video/x-raw, format=I420 ! appsink max-buffers=1 drop=true

Later, I convert the YUV_I420 to BGR with cvtColor. I figured I could just go from NV12 to BGR instead of this current method.

edit retag flag offensive close merge delete


looks like it's the equivalent of cvtColorTwoPlane

how do you acquire the nv12 imge ? how does it look like in memory ?

if it's inside a cv::Mat, you could try to split it into 2 ROIs, upper 2/3 height for Y, lower 1/3 for UV

berak gravatar imageberak ( 2020-10-24 01:57:39 -0500 )edit

Here is link for cv.cvtColorTwoPlane

supra56 gravatar imagesupra56 ( 2020-10-24 07:28:13 -0500 )edit

@berak Output image of the same row/column size and depth as ysrc

supra56 gravatar imagesupra56 ( 2020-10-24 07:31:10 -0500 )edit

^^ i asked about the input

berak gravatar imageberak ( 2020-10-24 08:28:45 -0500 )edit

Well I'm not really sure how to check because the gstreamer pipeline won't open since NV12 isn't a supported format in OpenCV. I figured the pipeline would only be able to give me one Mat which I would send into the function, but evidently that's not how the function works.

warpstar22 gravatar imagewarpstar22 ( 2020-10-26 11:15:46 -0500 )edit

I guess I'm partly confused as to the purpose of this function. It seems perfect to work with Nvidia's stuff so I was just assuming it would be easy for me to add to the gstreamer pipeline.

warpstar22 gravatar imagewarpstar22 ( 2020-10-26 11:16:47 -0500 )edit

it's also unclear what your "gstreamer pipeline" exactly is.

btw, most android phones seem to use nv12 natively, so this is not an "exotic problem"

berak gravatar imageberak ( 2020-10-26 11:33:30 -0500 )edit

Sorry, updated my question with the gstreamer pipeline I am using

warpstar22 gravatar imagewarpstar22 ( 2020-10-26 20:23:44 -0500 )edit