# solvePNP() and trinagulatePoints() do not give the anticipated results - Python

Hello guys. I'm doing my senior thesis and i have this problem for about a month. I'm a surveyor engineer , so i have measured specific points on the wall with the shape of crosshair. So they have specific coordinates X,Y,Z in this local coordinate system i have created. Now, i also have two cameras in this lab and as a first step i want to find X,Y,Z coordinates for some of these crosshairs in order to check if my algorithm runs properly. But the results from my algorithm doesn't match the results i have measured(ground truth values). These are images of the left and right cameras and i have circled points i used to check my results.

So i will explain a little bit my code.. When i run the code the susteAGE4 image is appear. I check the 80 , 5 , 67, 69 with this exact order. I click ESC. Then the sustAGE3 is appeared. I click the 40 , 55 , 59 , 66 with this order. These image coordinates combined with the world coordinate i have measured give me the translation and rotation vectors using solvePNP(). After that i click ESC again and i click 66,67,70,68 points, firsty in the left, after that in the right image in order to find the X,Y,Z using triangulatePoints() coordinates and compare them with my measured world coordinates. But the results doesn't match. I believe it's something with my projective matrices i create manually but i am not sure. When i run my code i firstly run imgp3 and imgp4 dicts so my code runs without errors. I also noticed that every time i run the code the results X Y Z of 66 67 70 68 are quite different. If someone needs the data in order to run the code i will gladly send them through email or something

Here is my code:

import numpy as np import cv2 import pandas as pd

def coords(event,x,y,flags,param=(imgp4,imgp3)): if event==cv2.EVENT_LBUTTONDOWN: global i global imgp4 global imgp3 i+=1

    if i<=4:
imgp4[i]=(x,y)
else:
global j
j+=1
imgp3[j]=(x,y)


def coords_test(event,x,y,flags,param): if event==cv2.EVENT_LBUTTONDOWN: global k global testL global testR k+=1

    if k<=4:
print(x,y)
testL[k]=(x,y)
else:
global l
l+=1
print(x,y)
testR[l]=(x,y)


def pose(Coords,imgp,mtx,dst):

# Find the rotation and translation vectors.

ret,rvecs, tvecs = cv2.solvePnP(Coords,imgp,mtx,dst)
return (rvecs,tvecs)


def kfw(rvecs,tvecs):

rotation_matrix=cv2.Rodrigues(rvecs)[0]

projection_matrix=np.append(rotation_matrix,tvecs,axis=1)
zyx=cv2.decomposeProjectionMatrix(projection_matrix)

return(rotation_matrix,projection_matrix,zyx)


if __name__=='__main__':

Coords=pd.read_csv("coords.txt",sep=",",header=None)

# Start Live video
video_left=cv2.VideoCapture('video4.mp4')
video_right=cv2.VideoCapture('video3.mp4')
i=0
j=0
k=0 ...
edit retag close merge delete