Using the IR image as a mask for RGB image

asked 2016-06-03 03:23:41 -0600

I'm passing the RGB, Depth and IR image through from libfreenect2 to opencv, and I am having trouble working out how to use the the IR image as a mask over the RGB image to remove everything except humans from a video stream.

I've gotten the images from libfreenect2 and converted them to the following image types (based off this post) :

    cv::Mat(rgb->height, rgb->width, CV_8UC4, rgb->data).copyTo(rgbmat);
    cv::Mat(ir->height, ir->width, CV_32FC1, ir->data).copyTo(irmat);
    cv::Mat(depth->height, depth->width, CV_32FC1, depth->data).copyTo(depthmat);

Thanks in advance!

answered 2016-06-03 04:01:15 -0600

FooBar
2938 ●9 ●37 http://magazino.eu/

updated 2016-06-03 04:41:42 -0600

Just pass the IR image as second parameter to copyTo. Only pixels with a non-zero value in the mask are copied:

CopyTo With mask

// Update: As the mask has to be CV_8U, you could use cv::convertTo to change the type.

edit flag offensive delete link

Comments

wait, masks have to be CV_8U, and you probably have to scan the value range of the ir image, and then threshold it at an approp. value)

berak ( 2016-06-03 04:11:41 -0600 )edit

Thanks for the hint!

FooBar ( 2016-06-03 04:41:17 -0600 )edit

Thanks for helping getting me started. So using this source code, it does compile and run but I think there is still an issue with the conversion.

cv::Mat(ir->height, ir->width, CV_32FC1, ir->data).copyTo(irmat);
irmat.convertTo(irmat, CV_8U, 1, 0); 
resize(irmat, irmat, cvSize(1920, 1080); 
cv::Mat(depth->height, depth->width, CV_32FC1, depth->data).copyTo(depthmat);
cv::Mat(rgb->height, rgb->width, CV_8UC4, rgb->data).copyTo(rgbmat, irmat);

When running this program, the IR image and the depth image when doing something similar don't match up properly to the rgb image.

carouselification ( 2016-06-03 05:49:31 -0600 )edit

did you find out, in which range your ir (float) values are ?

convertTo() clips/saturates values > 255 to 255, so if your range there is larger, you might need a different scale factor

berak ( 2016-06-03 06:30:48 -0600 )edit

Is there an easy way to calculate this scale factor in opencv?

carouselification ( 2016-06-04 01:18:54 -0600 )edit

minMaxLoc will give you the minimum/maximum value

berak ( 2016-06-04 01:53:08 -0600 )edit

add a comment

Using the IR image as a mask for RGB image

1 answer

Comments

Links

Question Tools

Stats

Related questions

Using the IR image as a mask for RGB image edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

Using the IR image as a mask for RGB image