Transforming Point Cloud to get Top Down image edit

asked 2016-10-24 19:06:05 -0600


My task : I have a task where i am asked to track parcels(carton boxes) of different dimensions moving on a conveyor. I am using Asus Xtion pro camera mounted on top of a conveyor in any inclined angle. I am looking for a model free object tracker that will detect boxes in the scene,track them & gives their 6DOF ? My target object is just a box and i want to eliminate all other things in the scene.

My approach :

Step 1) I do Point cloud preprocessing like downsampling,pass through filtering and segmentation. All these should give me a final point cloud containing only the objects on the conveyor.

Step 2) I planned to make the "z" values in each point(depth value) as zero, thereby making the point cloud of the box to be flat on the ground.

step 3) I planned to transfer the view of the camera from any inclined position to a top down view so that i can view any number of carton boxes moving on the conveyor from a top down view . I feel the top down view will prevent perpective viewing problems

The process flow of step 2 and 3 is shown below: image description

step 4) After the top down view of the point cloud is achieved, i need to convert the 3d point cloud to 2d image ,so that i can perform object tracking with so many OpenCV based tracking algorithms available.

A Sample point cloud is shown below in different views

Original View from camera:

image description

Point Cloud View 1:

image description

Point Cloud View 2:

image description

Point Cloud Target/Desired View for converting to 2d :(The box is the target.All the ground plane and unnecessary points would be eliminated)

image description

Is my approach correct? How will i achieve steps 2,3 and 4?

edit retag flag offensive close merge delete


imho, this is not an opencv question, but a pcl one, thus you should rather ask e.g. here

berak gravatar imageberak ( 2016-10-25 03:46:39 -0600 )edit

your top-down view with z=0 is still a point cloud, not a 2d image (where opencv would come in)

you also lost almost any shape / texture information

berak gravatar imageberak ( 2016-10-25 04:43:52 -0600 )edit

@berak My tracking is not texture based. So i just need a stream of images where there would be objects moving and i can track that. Do you have any idea on how to convert a point cloud to a mat Image

ramanan1991 gravatar imageramanan1991 ( 2016-10-27 07:23:44 -0600 )edit