# cv::Mat to PCL point cloud, to data ptr?

I have a cv::Mat that is a depth image.

I am converting it to a PCL pointcloud like this:

PointCloud::Ptr PointcloudUtils::RGBDtoPCL(cv::Mat depth_image, Eigen::Matrix3f& _intrinsics)
{
PointCloud::Ptr pointcloud(new PointCloud);

float fx = _intrinsics(0, 0);
float fy = _intrinsics(1, 1);
float cx = _intrinsics(0, 2);
float cy = _intrinsics(1, 2);

float factor = 1;

depth_image.convertTo(depth_image, CV_32F); // convert the image data to float type

if (!depth_image.data) {
std::cerr << "No depth data!!!" << std::endl;
exit(EXIT_FAILURE);
}

pointcloud->width = depth_image.cols; //Dimensions must be initialized to use 2-D indexing
pointcloud->height = depth_image.rows;
pointcloud->resize(pointcloud->width*pointcloud->height);

#pragma omp parallel for
for (int v = 0; v < depth_image.rows; v ++)
{
for (int u = 0; u < depth_image.cols; u ++)
{
float Z = depth_image.at<float>(v, u) / factor;

PointT p;
p.z = Z;
p.x = (u - cx) * Z / fx;
p.y = (v - cy) * Z / fy;

p.z = p.z / 1000;
p.x = p.x / 1000;
p.y = p.y / 1000;

pointcloud->points.push_back(p);

}
}

return pointcloud;

}


I am also getting the data from the Mat like this:

unsigned short* dataMat1 = depth_image.ptr<unsigned short>();


What i need to do now is Convert the pointcloud into an unsigned short * that matches the dataMat1 above.

What would this conversion look like?

Does the depth_image.ptr<unsigned short> just read row 0: col1, col2, col3.... row 1: col1,col2etc?

Thank you!

EDIT:

To simplify the problem, i am trying to get the pcl - to - Mat conversion working. I load my image, run the above code to get a pointcloud, then take that pointcloud and run:

cv::Mat PointcloudUtils::PCL2Mat(PointCloud::Ptr pointcloud, int original_width, int original_height)
{
cv::Mat depth_image(original_height, original_width, CV_32F);

//  depth_image.create

int count = 0;

#pragma omp parallel for
for (int v = 0; v < depth_image.rows; ++v)
{
for (int u = 0; u < depth_image.cols; ++u)
{
depth_image.at<float>(v, u) = pointcloud->points.at(count++).z * 1000;

}
}

normalize(depth_image, depth_image, 0, 255, cv::NORM_MINMAX, CV_8U);

return depth_image;

}


I pass that function:

cv::Mat newDepth;
newDepth = pointcloudUtils_.PCL2Mat(newCloud, depth_image.cols, depth_image.rows);
cv::imshow("new", newDepth);
cv::waitKey(1);


(where depth_image is the original loaded image). But When i view the newly created mat, it is solid black. Where am i going wrong?

thanks!

edit retag close merge delete

1

btw: if you use pointcloud->points.push_back(p);, you should not do: pointcloud->points.resize(pointcloud->height * pointcloud->width); else you push the points after the already allocated , but all empty points.

( 2017-10-12 07:47:07 -0600 )edit

Hi! thanks! That part seems to be working pretty well. I will pull that line out to test though. As to the conversion I need, are you able to point me in the right direction?

( 2017-10-12 07:52:43 -0600 )edit

tbh, i did not understand you there. do you really want to convert the pointcloud back to ushort ?

(i also vaguely remember, that PCL had some function to condense clouds to images)

( 2017-10-12 08:11:58 -0600 )edit
1

I do yup. Basically I create the cloud from the image, then process the cloud. Then I need to run it through a function that takes a Mat.ptr. So either I can try to convert the cloud back to a Mat and get the data.ptr, or just skip the middle step. If possible.

( 2017-10-12 08:14:58 -0600 )edit
1

"That part seems to be working pretty well." -- meaning, you see a nice cloud in the viz, i guess.

but if you take a look at cloud.points.size(), you'll see, it's 2x as large as it should be, and that the 1st half is all zeros

( 2017-10-12 09:44:32 -0600 )edit

Ah i see! I will make that change. thank you. :)

( 2017-10-12 09:57:49 -0600 )edit

@berak, If you have a minute, I have edited my question to add some conversion code. It looks correct to me, but doesn't work!

( 2017-10-13 03:25:23 -0600 )edit

Sort by » oldest newest most voted

"But When i view the newly created mat, it is solid black. "

ah, this is now easy to answer. idk. , in which range your z values are, but most likely, your conversion to u8 needs a scale factor, and maybe an offset (remember, you need [0..255] for u8).

check min/max vals before the conversion. (maybe you even got negative values ?)

double m,M;
minMaxLoc(depth_image, &m, &M, 0, 0);


then use that information like:

depth_image.convertTo(depth_image, CV_8U, 255.0/(M-m), -m);


alternatively, you could normalize your image (and change type on the way):

normalize(depth_image, depth_image, 0, 255, NORM_MINMAX, CV_8U);

more

Hi again! And thanks. :) i have tried replacing the convert with the normalize line above, and still see a black image... the same with adding the scale to the convert. weird.

( 2017-10-13 03:40:27 -0600 )edit

try normalize, it's the better idea. (and my initial formula was broken)

( 2017-10-13 03:47:35 -0600 )edit

Question edited with updated code. I still get a black image. I am checking the original image (looks good) and the cloud (looks good), but the converted Mat is black.

( 2017-10-13 03:51:06 -0600 )edit