# projectPoints fails with points behind the camera

I'm using the pojectPoints opencv function to get the projection of a 3d point in a camera image plane.

cv::projectPoints(inputPoint, rvec, tvec, fundamentalMatrix, distCoeffsMat, outputPoint);


The problem I'm facing is when Z (in camera local frame) is negative, instead of returning a point out of the image boundaries, it returns me the symmetric (Z positive) instead. I was expecting that function to check for positive Z values...

I can check this manually by myself, but is there a better way?

Thanks!

edit retag close merge delete

I encountered the same problem. What's worse is I have 3d points both in the front and the back of the camera.... Is there anyway I can solve this?

( 2014-05-22 19:18:11 -0500 )edit

Sort by ยป oldest newest most voted

You have to find the homogenous 2d points explicitly. Check their z-value. If negative ignore them. Those are the points behind you.

In python it looks like this:

R, t - current 6dof pose of the camera
K - 3x3 Camera matrix
D - distortion coefficients
xyz - Nx3 3d points

proj_mat = np.dot(K, np.hstack((R, t[:, np.newaxis])))
# convert 3D points into homgenous points
xyz_hom = np.hstack((xyzs, np.ones((xyzs.shape[0], 1))))

xy_hom = np.dot(proj_mat, xyz_hom.T).T

# get 2d coordinates in image [pixels]
z = xy_hom[:, -1]
xy = xy_hom[:, :2] / np.tile(z[:, np.newaxis], (1, 2))

# undistort - has to be 1xNx2 structure
xy = cv2.undistortPoints(np.expand_dims(xy, axis=0), np.eye(3), D).squeeze()

# drop all points behind camera
xy = xy[z > 0]

more

Official site

GitHub

Wiki

Documentation