I am trying to generate key-points in a set of views given a number of 3D points, the extrinsic parameters of each view and the intrinsic parameters of the camera. However, I noticed that points do not lie within the size of the frame (e.g. 640 x 480). This is the data I am using:
- Camera pose (1st view) as a 6D vector (orientation + position in world coordinate frame): [-90 90 0 | 80 45 45];
- 3D point: [50 50 40]
- Image: 640x480 pxs
- Focal length: 30 mm
- Sensor size: 22x16 mm (Sw x Sh)
Thus, the camera matrix (intrinsic parameters) becomes: [fIw/Sw 0 Iw/2; 0 fIh/Sh Ih/2; 0 0 1];
When applying the formula for the pinhole camera model:
f = K * R * [I|t] * M
where R and t are the rotation matrix and the translation vector, respectively - coming from the camera pose - and M is in homogeneous coordinates in this case, I cannot obtain a point within the frame size (i.e. 640 x 480). Please notice that is already scaled by its 3rd component to have a 2D point.
Do you have any idea why the projection does not work? When visualizing the point and the camera, I am already sure the point is in front of the camera in the world coordinate system.
Can you confirm that the identity matrix for the rotation corresponds to the camera looking upwards, please?
I also tried the function cv::projectPoints() to verify, but return an error probably connected with the fact the depth of the point is not positive in the camera coordinate system.