Triangulation scaling problem. (Multiple Images)

edit

Hello everyone,

I try to do 3d metric reconstruction with 60 images.

I use python and there is only this function in opencv to triangulation with python. It reconstructs the images two by two.

There are 2 methods that I consider using to make triangulation.

The first method is to select 1 image as base and triangulate one by one with its neighbors ;

Like this:

where 1 -> Base Image ; 2,3,4,5,6 -> Base Image's neighbors

 R1 =[[1 0 0]   t1 = [[0]]
      [0 1 0]         [0]
      [0 0 1]]        [0]]

 R2,t2 = cv2.findEssentialMat(img1.pts, img2.pts, K) and cv2.recoverPose(E)
 R3,t3 = cv2.findEssentialMat(img1.pts, img3.pts, K) and cv2.recoverPose(E)
 .
 .
 R6,t6 = cv2.findEssentialMat(img1.pts, img6.pts, K) and cv2.recoverPose(E)


P1 = K[R1|t1]  
P2 = K[R2|t2]  
.  
.  
P6 = K[R6|t6]    

3d_first = cv2.triangulatePoints(P1,P2, pts1,pts2)  
3d_second = cv2.triangulatePoints(P1,P3, pts1,pts3)  
3d_third = cv2.triangulatePoints(P1,P4, pts1,pts4)  
3d_fourth = cv2.triangulatePoints(P1,P5, pts1,pts5)  
3d_fifth = cv2.triangulatePoints(P1,P6, pts1,pts6)

When I visualize the 3d points, they do not overlap and there is a difference between them.(I guess It is a scaling problem).

How can i fix the problem?

There is a suggestion like that ;

t1 = t1 * (camera1 and camera2 baseline (unit metric))  
t2 = t2 * (camera1 and camera3 baseline (unit metric))

but this sometimes does not work. Does this give me a metric results?

The second method is to triangulate the photos as img1-img2, img2-img3, img3-img4 instead of img1-img2, img1-img3, img1-img4. In this case, I have to bring all the 3D points to the same plane. Because P1 = [I | 0] for img1-img2 triangulation, P2 = [I, 0] for img2-img3 triangulation.

In this case, point clouds for img1-img2 will appear on the plane img1, and point clouds for img2, img3 will appear on the plane 2.

How can I bring them to the same plane?

Thanks in advance.