Ask Your Question

World coordinate space from camera coordinate space?

asked 2013-11-15 21:24:15 -0500

J gravatar image

I am doing structure from motion from multiple images using OpenCV. I have 8 images, and I have generated point clouds for each pair of images (img1&2, img2&3, img3&4 etc.). I know that each individual point cloud is correct because they look good when displayed in VTK / OpenGL.

I want to combine all of the point clouds to get a denser reconstruction. From what I understand, all my point clouds right now are on a different camera coordinate system. I am trying to use cv2.solvePnP to compute my camera poses and transform my 3D points with the inverse of the camera pose so that my point clouds are all on the same coordinate system, but the points from each successive 3D point set still have coordinates that are orders of magnitude away from those of the previous set.

Here is my code:

images = ['00.jpg', '01.jpg', '02.jpg', '03.jpg']
cam_poses = []

for i in range(len(images)-1):
    K, pts_3D, img_pts = gen_pt_cloud(i, images[i], images[i+1])

    if i == 0:
        # first 2 images
        pt_cloud = np.array(pts_3D)
        colours = np.array(img_colours)

    elif i >= 1:
        # get the pose of the camera
        rvec, tvec = cv2.solvePnP(pts_3D, img_pts, K, None)[1:3]
        pose = np.vstack((np.hstack((cv2.Rodrigues(rvec)[0], tvec)), np.array([0,0,0,1])))

        # transform the 3D points using the camera pose
        for j in reversed(range(i)):
            pts_3D = np.array([[j]), pts_3D) ])[0]
            pt_cloud = np.vstack((pt_cloud, pts_3D))


Is there something I'm doing wrong?

Also, I checked the OpenCV documentation and saw that objectPoints is an array of object points in the object coordinate space. What does that mean? My objectPoints (pts_3D) are on their specific camera coordinate system. Is that why my points are off? If it is, what should I do to transform the objectPoints so that they are in the object coordinate space?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2013-11-17 01:31:47 -0500

Robst gravatar image


objectPoints are points expressed in the object coordinate space, for example, for a chess calibration pattern, we can fix a coordinate system in one of the corners of the plane so that the chess board is on the x-y plane an all chess corners have a z-coordinate value of zero. You would supply to the solvePnP function the coordinates of the chess corners expressed in this coordinate system. Notice that for this set of points we haven't used the camera coordinate system.

To transform your current objectPoints to the object coordinate system you have to find the transformation from the camera coordinate system to the object coordinate system. solvePnP will be handy for this. Also, take into account that if you obtain coordinate system C2 by translating coordinate system C1 by T and rotating it by a rotation matrix R , the inverse transformation is given by a rotation by R^t (the transpose) and a translation -RT.

Hope this helps.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools


Asked: 2013-11-15 21:24:15 -0500

Seen: 2,221 times

Last updated: Nov 17 '13