Triangulation scaling problem. (Multiple Images)

Hello everyone,

I try to do 3d metric reconstruction with 60 images.

I use python and there is only this function in opencv to triangulation with python. It reconstructs the images two by two.

There are 2 methods that I consider using to make triangulation.

The first method is to select 1 image as base and triangulate one by one with its neighbors ;

Like this:

where 1 -> Base Image ; 2,3,4,5,6 -> Base Image's neighbors

 R1 =[[1 0 0]   t1 = [[0]]
[0 1 0]         [0]
[0 0 1]]        [0]]

R2,t2 = cv2.findEssentialMat(img1.pts, img2.pts, K) and cv2.recoverPose(E)
R3,t3 = cv2.findEssentialMat(img1.pts, img3.pts, K) and cv2.recoverPose(E)
.
.
R6,t6 = cv2.findEssentialMat(img1.pts, img6.pts, K) and cv2.recoverPose(E)

P1 = K[R1|t1]
P2 = K[R2|t2]
.
.
P6 = K[R6|t6]

3d_first = cv2.triangulatePoints(P1,P2, pts1,pts2)
3d_second = cv2.triangulatePoints(P1,P3, pts1,pts3)
3d_third = cv2.triangulatePoints(P1,P4, pts1,pts4)
3d_fourth = cv2.triangulatePoints(P1,P5, pts1,pts5)
3d_fifth = cv2.triangulatePoints(P1,P6, pts1,pts6)


When I visualize the 3d points, they do not overlap and there is a difference between them.(I guess It is a scaling problem).

How can i fix the problem?

There is a suggestion like that ;

t1 = t1 * (camera1 and camera2 baseline (unit metric))
t2 = t2 * (camera1 and camera3 baseline (unit metric))


but this sometimes does not work. Does this give me a metric results?

The second method is to triangulate the photos as img1-img2, img2-img3, img3-img4 instead of img1-img2, img1-img3, img1-img4. In this case, I have to bring all the 3D points to the same plane. Because P1 = [I | 0] for img1-img2 triangulation, P2 = [I, 0] for img2-img3 triangulation.

In this case, point clouds for img1-img2 will appear on the plane img1, and point clouds for img2, img3 will appear on the plane 2.

How can I bring them to the same plane?