VideoCapture from CSI camera is way off

asked 2019-04-09 20:45:10 -0600

NP41 gravatar image

I am using OpenCV 3.1 with CUDA enabled on the Tegra K1 with the OV5640 CSI Camera.

I tried for a long time to get Gstreamer working with VideoCapture to no avail. I tried multiple different pipelines and not one of them managed to open the camera so I used the default VideoCapture constructor. My code is as follows:

    cv::VideoCapture cap(0);
if (!cap.isOpened()){
    printf("Camera not opened \n");
    return -1;
}

// View video
cv::Mat frame;

for(int i = 0; i < 1500; i++) {

    cap >> frame;

    imshow("Display window", frame);

    cv::waitKey(1); 

}

and my output is as follows: image description

At first, I thought it was the Bayer output but after some investigating the image stream has the following properties: 640x480 resolution

3-channel image

CV_16U3 - 16 bit encoding

Since I do not know the colorspace the image is in, I tried to convert to every possible colorspace with cvtColor() and every transformation ended up making the image worse.

Any help would be greatly appreciated. I was initially trying to get the images in YUV or YCrCb colorspace but HSV or LAB would also work just fine.

edit retag flag offensive close merge delete

Comments

I also tried this bit of code from another forum post

    // Copy the data into an OpenCV Mat structure
    cv::Mat mat16uc1_bayer(480, 640, CV_16UC1, frame.data); //what is input buffer?

    // Decode the Bayer data to RGB but keep using 16 bits per channel
    cv::Mat mat16uc3_rgb(640, 480, CV_16UC3);
    cv::cvtColor(mat16uc1_bayer, mat16uc3_rgb, cv::COLOR_BayerGR2RGB);

    // Convert the 16-bit per channel RGB image to 8-bit per channel
    cv::Mat mat8uc3_rgb(640, 480, CV_8UC3);
    mat16uc3_rgb.convertTo(mat8uc3_rgb, CV_8UC3, 1.0/256);

which did not work either, the image became more distorted

NP41 gravatar imageNP41 ( 2019-04-09 20:45:53 -0600 )edit

Do other camera application work? Try gstreamer pipelines via gst-launch-1.0 app: gst-launch-1.0 v4l2src ! xvimagesink.

mshabunin gravatar imagemshabunin ( 2019-04-10 07:57:31 -0600 )edit

Hi, What do you mean by other camera applications? I can gst-launch to get clear video data. The command I use is: gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-raw,format=UYVY, width=1920,height=1080' ! xvimagesink

NP41 gravatar imageNP41 ( 2019-09-09 17:58:58 -0600 )edit

Try VideoCapture cap("v4l2src device="/dev/video0" ! 'video/x-raw,format=UYVY, width=1920,height=1080' ! appsink", CAP_GSTREAMER); and VideoCapture cap(0, CAP_V4L);

mshabunin gravatar imagemshabunin ( 2019-09-20 06:05:22 -0600 )edit
add a comment