How to get 2d correspondances for 3d cad model points for non textured objects?

asked 2020-06-30 09:44:48 -0600

astronaut gravatar image


I have to estimate a very accurate 6D Pose of regular shape objects (like blocks for construction ) without any textured. I have the 3D CAD model of the object, and I know roughly the coordinates of the blocks. So the camera is pointing down to the object and can use stereo cameras or ZED

I know roughly the coordinates on the object and the camera will be placed around 1m away from the object. I would like to use OpenCV solvePnP method and for that need to find the 2d correspondences of the scene image for 3D CAD model points of the objects.

I search and found this tutorial for Real Time pose estimation of a textured object but in my case the objects are nontextured. So I can not use such key descriptors like SURF or SIFT or ORB. And also I would like to use it in real-time and as input to take my camera input.

Hence, research in this area has focused on tracking using natural features, such as edge-based, optical flow-based,template-based, and keypoint-based. Is this relevant so I can find the 2d correspondences for 3d cad model points for non-textured objects like construction blocks and can get the 6D Pose?

Also, I know that exists Harris Corners, Hough Lines, Canny Edge, or Fast Corners descriptors but not sure if using them can track the object and find the 2D-3D points correspondences. Can I use them in Real Time pose estimation of a textured object and modify the source code for real-time so I can track my non-textured objects? If yes what should I change in the source code and how to run it in real-time?

Any help would appreciate.

edit retag flag offensive close merge delete