Ask Your Question
0

realsense R200, difference between saved images and live stream?

asked 2017-02-16 14:06:27 -0600

stillNovice gravatar image

updated 2017-02-16 14:33:36 -0600

Hi, I have a realsense r200 camera, and am using it with Stereo Odometry.

The images are converted to cv::mat like this:

Mat lFrame(480, 640, CV_16U, (uchar *)dev->get_frame_data(rs::stream::infrared));

which looks fine, and is from the docs.

When i save images out using:

cv::imwrite(name, img);

and run them through the third party Odometry library by loading the images, like this:

cv::Mat left_img = cv::imread(left_img_file_name.c_str(), 0); // load as black and white image
uint8_t* left_img_data = left_img.data;

Everything works great. BUT, when i use a live stream of images, like this:

Mat lFrame(480, 640, CV_16U, (uchar *)dev->get_frame_data(rs::stream::infrared));
uint8_t* left_img_data = lFrame.data;

It does not work at all.

I suspect this is due to a mismatch in Mat types, but I am lost.

I have tried:

cv::cvtColor(lFrame, leftImg, CV_GRAY2RGB);

cv::cvtColor(lFrame, leftImg, CV_GRAY2BGR);

But I see no difference. Does anyone have any thoughts on what might be missing here?

..if it helps.. I have found an example by the author of the library. They used:

IplImage *I = cvCreateImageHeader(cvSize(dc_frame->size[0],dc_frame->size[1]),IPL_DEPTH_8U,1);
      cvSetData(I,dc_frame->image,dc_frame->size[0]);

Thank you!

edit retag flag offensive close merge delete

Comments

run them through the third party Odometry library then your best guess is that you just get this solved with guys using that library. Not really sure if this is the best place to do so.

StevenPuttemans gravatar imageStevenPuttemans ( 2017-02-17 04:19:43 -0600 )edit

Hi, Thanks for your response. I am just wondering if there is anything obvious that it could be. Is there any obvious difference in the Mat types from the example code to the code I am using? Is there any other 16 bit Mat type I could try? Thanks again!

stillNovice gravatar imagestillNovice ( 2017-02-17 04:57:16 -0600 )edit

@stillNovice, is your project open source? Can you provide a link?

Der Luftmensch gravatar imageDer Luftmensch ( 2017-02-17 14:32:45 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2017-02-17 09:04:31 -0600

Eduardo gravatar image

updated 2017-02-17 09:17:39 -0600

If what you want is to get an infrared image in CV_8U, you have two options for me:

  • request directly an infrared image at this format: dev->enable_stream(rs::stream::infrared, 640, 480, rs::format::y8, 30);
  • just convert the native CV_16U infrared image to the wanted format if you cannot change the stream for whatever reason (you need to scale to the destination range, otherwise the image is saturated):

    Mat infrared_8U;
    infrared.convertTo(infrared_8U, CV_8U, 1/256.0);
    

You can also directly shift the bits:

Mat infrared_8U(infrared.size(), CV_8U);
for (int i = 0 ;i < infrared_8U.rows; i++) {
  for (int j = 0; j < infrared_8U.cols; j++) {
    infrared_8U.at<uchar>(i,j) = infrared.at<ushort>(i,j) >> 8;
  }
}

Not sure what happens if you write a CV_16U image on disk as you seem to do.

Supported formats for each RealSense camera can be found here. Getting started for the librealsense and OpenCV: https://github.com/IntelRealSense/librealsense/blob/master/doc/stepbystep/getting_started_with_openCV.md.

Demo code tested with an Intel SR300 camera:

// Create a context object. This object owns the handles to all connected realsense devices
rs::context ctx;

// Access the first available RealSense device
rs::device * dev = ctx.get_device(0);

// Configure Infrared stream to run at VGA resolution at 30 frames per second
dev->enable_stream(rs::stream::infrared, 640, 480, rs::format::y16, 30);

// Start streaming
dev->start();

// Camera warmup - Dropped several first frames to let auto-exposure stabilize
for(int i = 0; i < 30; i++) {
  dev->wait_for_frames();
}

// Creating OpenCV Matrix from a color image
Mat infrared(Size(640, 480), CV_16U, (void*)dev->get_frame_data(rs::stream::infrared), Mat::AUTO_STEP);

Mat infrared_8U;
infrared.convertTo(infrared_8U, CV_8U, 1/256.0);

// Display in a GUI
namedWindow("Display Image", WINDOW_AUTOSIZE );
imshow("Display Image", infrared_8U);

waitKey(0);
edit flag offensive delete link more

Comments

Thank you! The conversion to 8bit first solved it for me.

stillNovice gravatar imagestillNovice ( 2017-02-17 13:39:52 -0600 )edit
0

answered 2017-02-17 07:31:31 -0600

Mat lFrame(480, 640, CV_16U, (uchar *)dev->get_frame_data(rs::stream::infrared));

For me this seems weird. You have a 16 bit output image that you want, but you are jumping in memory with a uchar pointer, which is typically for 8 bit. Shouldn't it be at least Mat lFrame(480, 640, CV_16U, (float *)dev->get_frame_data(rs::stream::infrared)); to get 16 bit pointers?

edit flag offensive delete link more

Comments

1

Thank you for taking the time to look at this. It was as simple as converting to 8bit. .convertTo(infrared_8U, CV_8U, 1/256.0); Gotta love an easy fix. :) Thanks again.

stillNovice gravatar imagestillNovice ( 2017-02-17 10:29:16 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-02-16 14:06:27 -0600

Seen: 137 times

Last updated: Feb 17