# World coordinate space from camera coordinate space?

I am doing structure from motion from multiple images using OpenCV. I have 8 images, and I have generated point clouds for each pair of images (img1&2, img2&3, img3&4 etc.). I know that each individual point cloud is correct because they look good when displayed in VTK / OpenGL.

I want to combine all of the point clouds to get a denser reconstruction. From what I understand, all my point clouds right now are on a different camera coordinate system. I am trying to use cv2.solvePnP to compute my camera poses and transform my 3D points with the inverse of the camera pose so that my point clouds are all on the same coordinate system, but the points from each successive 3D point set still have coordinates that are orders of magnitude away from those of the previous set.

Here is my code:

images = ['00.jpg', '01.jpg', '02.jpg', '03.jpg']
cam_poses = []

for i in range(len(images)-1):
K, pts_3D, img_pts = gen_pt_cloud(i, images[i], images[i+1])

if i == 0:
# first 2 images
pt_cloud = np.array(pts_3D)
colours = np.array(img_colours)

elif i >= 1:
# get the pose of the camera
rvec, tvec = cv2.solvePnP(pts_3D, img_pts, K, None)[1:3]
pose = np.vstack((np.hstack((cv2.Rodrigues(rvec)[0], tvec)), np.array([0,0,0,1])))
cam_poses.append(pose)

# transform the 3D points using the camera pose
for j in reversed(range(i)):
pts_3D = np.array([ np.dot(np.linalg.inv(cam_poses[j]), pts_3D) ])[0]
pt_cloud = np.vstack((pt_cloud, pts_3D))

vtk_cloud.vtk_show_points(pt_cloud)


Is there something I'm doing wrong?

Also, I checked the OpenCV documentation and saw that objectPoints is an array of object points in the object coordinate space. What does that mean? My objectPoints (pts_3D) are on their specific camera coordinate system. Is that why my points are off? If it is, what should I do to transform the objectPoints so that they are in the object coordinate space?

edit retag close merge delete

Sort by » oldest newest most voted

Hello:

objectPoints are points expressed in the object coordinate space, for example, for a chess calibration pattern, we can fix a coordinate system in one of the corners of the plane so that the chess board is on the x-y plane an all chess corners have a z-coordinate value of zero. You would supply to the solvePnP function the coordinates of the chess corners expressed in this coordinate system. Notice that for this set of points we haven't used the camera coordinate system.

To transform your current objectPoints to the object coordinate system you have to find the transformation from the camera coordinate system to the object coordinate system. solvePnP will be handy for this. Also, take into account that if you obtain coordinate system C2 by translating coordinate system C1 by T and rotating it by a rotation matrix R , the inverse transformation is given by a rotation by R^t (the transpose) and a translation -RT.

Hope this helps.

more

Official site

GitHub

Wiki

Documentation