OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Mon, 12 Sep 2016 07:26:48 -0500What does projection matrix provided by the calibration represent?http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/I'm using such a tool from ROS/OpenCV in order to perform the [camera calibration](http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration). The procedure ends up providing: camera matrix, distortion parameters, rectification matrix and projection matrix.
As far as I know the projection matrix contains the intrinsic parameter matrix of the camera multiplied by the extrinsic parameters matrix of the matrix.
The extrinsic parameter matrix itself provides the roto-translation of the camera frame with respect to the world frame.
If these assumptions are correct...how the projection matrix is computed by Opencv? I,m not defining any world frame!
camera matrix
414.287922 0.000000 382.549277
0.000000 414.306025 230.875006
0.000000 0.000000 1.000000
distortion
-0.278237 0.063338 -0.001382 0.000732 0.000000
rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000
projection
297.051453 0.000000 387.628900 0.000000
0.000000 369.280731 227.051305 0.000000
0.000000 0.000000 1.000000 0.000000Thu, 16 Jan 2014 10:44:50 -0600http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/Answer by windonground for <p>I'm using such a tool from ROS/OpenCV in order to perform the <a href="http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration">camera calibration</a>. The procedure ends up providing: camera matrix, distortion parameters, rectification matrix and projection matrix.
As far as I know the projection matrix contains the intrinsic parameter matrix of the camera multiplied by the extrinsic parameters matrix of the matrix.
The extrinsic parameter matrix itself provides the roto-translation of the camera frame with respect to the world frame.
If these assumptions are correct...how the projection matrix is computed by Opencv? I,m not defining any world frame!</p>
<p>camera matrix
414.287922 0.000000 382.549277
0.000000 414.306025 230.875006
0.000000 0.000000 1.000000</p>
<p>distortion
-0.278237 0.063338 -0.001382 0.000732 0.000000</p>
<p>rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000</p>
<p>projection
297.051453 0.000000 387.628900 0.000000
0.000000 369.280731 227.051305 0.000000
0.000000 0.000000 1.000000 0.000000</p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?answer=61459#post-id-61459The meanning is shown in the definition of sesor_msgs/CameraInfo.msg in ros API: http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html
P is needed in this type of msg.
Projection/camera matrix
[fx' 0 cx' Tx]
P = [ 0 fy' cy' Ty]
[ 0 0 1 0]
By convention, this matrix specifies the intrinsic (camera) matrix
of the processed (rectified) image. That is, the left 3x3 portion
is the normal camera intrinsic matrix for the rectified image.
It projects 3D points in the camera coordinate frame to 2D pixel
coordinates using the focal lengths (fx', fy') and principal point
(cx', cy') - these may differ from the values in K.
For monocular cameras, Tx = Ty = 0. Normally, monocular cameras will
also have R = the identity and P[1:3,1:3] = K.
For a stereo pair, the fourth column [Tx Ty 0]' is related to the
position of the optical center of the second camera in the first
camera's frame. We assume Tz = 0 so both cameras are in the same
stereo image plane. The first camera always has Tx = Ty = 0. For
the right (second) camera of a horizontal stereo pair, Ty = 0 and
Tx = -fx' * B, where B is the baseline between the cameras.
Given a 3D point [X Y Z]', the projection (x, y) of the point onto
the rectified image is given by:
[u v w]' = P * [X Y Z 1]'
x = u / w
y = v / w
This holds for both images of a stereo pair.
It says in monocamera, for rectified image, P is [K | 0]; for raw image P is [K' | 0], the parameter of K' may differ from K, which can be thought as computed by projecting 3D points to rectified images. I guess K' is computed from projecting 3D points to raw images! Fri, 08 May 2015 01:21:49 -0500http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?answer=61459#post-id-61459Comment by Mehdi for <p>The meanning is shown in the definition of sesor_msgs/CameraInfo.msg in ros API: <a href="http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html">http://docs.ros.org/api/sensor_msgs/h...</a> <br/>
P is needed in this type of msg.</p>
<p>Projection/camera matrix <br/>
[fx' 0 cx' Tx] <br/>
P = [ 0 fy' cy' Ty] <br/>
[ 0 0 1 0] <br/>
By convention, this matrix specifies the intrinsic (camera) matrix <br/>
of the processed (rectified) image. That is, the left 3x3 portion <br/>
is the normal camera intrinsic matrix for the rectified image. <br/>
It projects 3D points in the camera coordinate frame to 2D pixel <br/>
coordinates using the focal lengths (fx', fy') and principal point <br/>
(cx', cy') - these may differ from the values in K. <br/>
For monocular cameras, Tx = Ty = 0. Normally, monocular cameras will <br/>
also have R = the identity and P[1:3,1:3] = K. <br/>
For a stereo pair, the fourth column [Tx Ty 0]' is related to the <br/>
position of the optical center of the second camera in the first <br/>
camera's frame. We assume Tz = 0 so both cameras are in the same <br/>
stereo image plane. The first camera always has Tx = Ty = 0. For <br/>
the right (second) camera of a horizontal stereo pair, Ty = 0 and <br/>
Tx = -fx' * B, where B is the baseline between the cameras. <br/>
Given a 3D point [X Y Z]', the projection (x, y) of the point onto <br/>
the rectified image is given by: <br/>
[u v w]' = P * [X Y Z 1]' <br/>
x = u / w <br/>
y = v / w <br/>
This holds for both images of a stereo pair. </p>
<p>It says in monocamera, for rectified image, P is [K | 0]; for raw image P is [K' | 0], the parameter of K' may differ from K, which can be thought as computed by projecting 3D points to rectified images. I guess K' is computed from projecting 3D points to raw images! </p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=102193#post-id-102193at the end you mean [K' | 0] for rectified image and [K | 0] for raw image right? That is what is written in the ros message comments.Mon, 12 Sep 2016 07:26:48 -0500http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=102193#post-id-102193Answer by jensenb for <p>I'm using such a tool from ROS/OpenCV in order to perform the <a href="http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration">camera calibration</a>. The procedure ends up providing: camera matrix, distortion parameters, rectification matrix and projection matrix.
As far as I know the projection matrix contains the intrinsic parameter matrix of the camera multiplied by the extrinsic parameters matrix of the matrix.
The extrinsic parameter matrix itself provides the roto-translation of the camera frame with respect to the world frame.
If these assumptions are correct...how the projection matrix is computed by Opencv? I,m not defining any world frame!</p>
<p>camera matrix
414.287922 0.000000 382.549277
0.000000 414.306025 230.875006
0.000000 0.000000 1.000000</p>
<p>distortion
-0.278237 0.063338 -0.001382 0.000732 0.000000</p>
<p>rectification
1.000000 0.000000 0.000000
0.000000 1.000000 0.000000
0.000000 0.000000 1.000000</p>
<p>projection
297.051453 0.000000 387.628900 0.000000
0.000000 369.280731 227.051305 0.000000
0.000000 0.000000 1.000000 0.000000</p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?answer=26603#post-id-26603You are correct that the projection matrix P (dimensions 3 x 4) contains the extrinsic and intrinsic camera parameters and is intended to be used to map homogenous world coordinates (4 x 1) to homogeneous camera coordinates (3 x 1). When performing calibration of a single camera, constructing a projection matrix for the camera doesn't make much sense (since the calibration procedure has no way to determine the relation to an arbitrary world frame), and unsurprisingly the underlying [OpenCV calibration function](http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#calibratecamera) does not return one. The reason you see a projection matrix being created with the ROS calibration tool is that ROS is simply returning a [CameraInfo](http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html) message. This message is meant for passing information about arbitrary cameras (not just during calibration), where it makes sense to include information a camera's extrinsics. In your case the Matrix should be P = [K | 0], since your camera should initially have no orientation and no translation to world frame.Thu, 16 Jan 2014 15:07:13 -0600http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?answer=26603#post-id-26603Comment by matteo for <p>You are correct that the projection matrix P (dimensions 3 x 4) contains the extrinsic and intrinsic camera parameters and is intended to be used to map homogenous world coordinates (4 x 1) to homogeneous camera coordinates (3 x 1). When performing calibration of a single camera, constructing a projection matrix for the camera doesn't make much sense (since the calibration procedure has no way to determine the relation to an arbitrary world frame), and unsurprisingly the underlying <a href="http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#calibratecamera">OpenCV calibration function</a> does not return one. The reason you see a projection matrix being created with the ROS calibration tool is that ROS is simply returning a <a href="http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html">CameraInfo</a> message. This message is meant for passing information about arbitrary cameras (not just during calibration), where it makes sense to include information a camera's extrinsics. In your case the Matrix should be P = [K | 0], since your camera should initially have no orientation and no translation to world frame.</p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=26667#post-id-26667if K stands for intrinsic matrix...no..in my case that's not true. i have just added the results of the calibration in the post.Fri, 17 Jan 2014 09:41:26 -0600http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=26667#post-id-26667Comment by jensenb for <p>You are correct that the projection matrix P (dimensions 3 x 4) contains the extrinsic and intrinsic camera parameters and is intended to be used to map homogenous world coordinates (4 x 1) to homogeneous camera coordinates (3 x 1). When performing calibration of a single camera, constructing a projection matrix for the camera doesn't make much sense (since the calibration procedure has no way to determine the relation to an arbitrary world frame), and unsurprisingly the underlying <a href="http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#calibratecamera">OpenCV calibration function</a> does not return one. The reason you see a projection matrix being created with the ROS calibration tool is that ROS is simply returning a <a href="http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html">CameraInfo</a> message. This message is meant for passing information about arbitrary cameras (not just during calibration), where it makes sense to include information a camera's extrinsics. In your case the Matrix should be P = [K | 0], since your camera should initially have no orientation and no translation to world frame.</p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=26670#post-id-26670The convention for the projection matrix is P = K [ R | t ], you will find this definition all the standard references (see Hartley and Zisserman p. 156 eg. 6.8). In the case of mono camera R should be the identity and t a null vector, resulting in P = K [ I | 0 ] = [K | 0]. Note that camera matrix K can be scaled with an arbitrary scale factor, so that you get more or less visible image after undistortion. I cannot see a simple scaling between the camera matrix and the projection matrix, something doesn't seem right with the values you posted. Since the calibration is through ROS. I suggest asking on their question site, after all OpenCV does not return a P matrix as part of the monocular calibration.Fri, 17 Jan 2014 11:53:28 -0600http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=26670#post-id-26670Comment by Mehdi for <p>You are correct that the projection matrix P (dimensions 3 x 4) contains the extrinsic and intrinsic camera parameters and is intended to be used to map homogenous world coordinates (4 x 1) to homogeneous camera coordinates (3 x 1). When performing calibration of a single camera, constructing a projection matrix for the camera doesn't make much sense (since the calibration procedure has no way to determine the relation to an arbitrary world frame), and unsurprisingly the underlying <a href="http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#calibratecamera">OpenCV calibration function</a> does not return one. The reason you see a projection matrix being created with the ROS calibration tool is that ROS is simply returning a <a href="http://docs.ros.org/api/sensor_msgs/html/msg/CameraInfo.html">CameraInfo</a> message. This message is meant for passing information about arbitrary cameras (not just during calibration), where it makes sense to include information a camera's extrinsics. In your case the Matrix should be P = [K | 0], since your camera should initially have no orientation and no translation to world frame.</p>
http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=102190#post-id-102190P is quite confusing here, the answer from @windonground is more correct. To reproject points from camera frame to pixels, you should use different matirces depending on the image you want to project on (raw distorted image or rectified image).Mon, 12 Sep 2016 07:03:35 -0500http://answers.opencv.org/question/26596/what-does-projection-matrix-provided-by-the-calibration-represent/?comment=102190#post-id-102190