Transforming Point Cloud to get Top Down image edit

edit

Hi

My task : I have a task where i am asked to track parcels(carton boxes) of different dimensions moving on a conveyor. I am using Asus Xtion pro camera mounted on top of a conveyor in any inclined angle. I am looking for a model free object tracker that will detect boxes in the scene,track them & gives their 6DOF ? My target object is just a box and i want to eliminate all other things in the scene.

My approach :

Step 1) I do Point cloud preprocessing like downsampling,pass through filtering and segmentation. All these should give me a final point cloud containing only the objects on the conveyor.

Step 2) I planned to make the "z" values in each point(depth value) as zero, thereby making the point cloud of the box to be flat on the ground.

step 3) I planned to transfer the view of the camera from any inclined position to a top down view so that i can view any number of carton boxes moving on the conveyor from a top down view . I feel the top down view will prevent perpective viewing problems

The process flow of step 2 and 3 is shown below:

step 4) After the top down view of the point cloud is achieved, i need to convert the 3d point cloud to 2d image ,so that i can perform object tracking with so many OpenCV based tracking algorithms available.

A Sample point cloud is shown below in different views

Original View from camera:

Point Cloud View 1:

Point Cloud View 2:

Point Cloud Target/Desired View for converting to 2d :(The box is the target.All the ground plane and unnecessary points would be eliminated)

Is my approach correct? How will i achieve steps 2,3 and 4?