OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Wed, 14 Jun 2017 03:51:42 -0500Compute fundamental matrix from camera calibrationhttp://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/Hello,
I try to compute the fundamental matrix given the following camera calibration parameters:
- camera Matrix 1/2 (camMat1, camMat2)
- rotation Vector 1/2 (rotVec1, rotVec2)
- translation Vector 1/2 (transVec1, transVec2)
According to the following formula the fundamental matrix F is computed by:
F = inverse(transpose(camMat1)) * R * S * inverse(camMat2)
Anyway, i am quite a bit lost how to compute R and S. I know that R is the rotation matrix which brings image 1 into image 2. Also i know that S is the translation vector to transform image 1 into image 2. My plan would be:
1) Rodrigues both rotation Vectors and substract rotation Matrix 1 from rotation Matrix 2
cv::Mat rotMat1, rotMat2;
cv::Rodrigues(rotVec1[0], rotMat1);
cv::Rodrigues(rotVec2[0], rotMat2);
cv::Mat R = rotMat2 - rotMat1;
2) Substract translation Vector 1 from translation Vector 2
T = transVec2 - transVec1
3) Compose S
S = [0,-T[3], T[2]; T[3], 0, -T[1]; -T[2], T[1], 0];
Would this be correct? Any help on this topic would be appreciated.
I hope this is not offtopic since it is not directly related to OpenCV.
*Edit: I worked in the solution you provided. I cross checked it with cv::stereoCalib(). Unfortunatly the matrizes does not match. Any suggestions what i did wrong?*
cv::Mat computeFundMat(cv::Mat camMat1, cv::Mat camMat2, vector<cv::Mat> rotVec1,
vector<cv::Mat> rotVec2, vector<cv::Mat> transVec1, vector<cv::Mat> transVec2)
{
cv::Mat rotMat1(3, 3, CV_64F), rotMat2(3, 3, CV_64F);
cv::Mat transVec2toWorld(3, 1, CV_64F);
cv::Mat R(3, 3, CV_64F), T(3, 1, CV_64F), S(cv::Mat::zeros(3, 3, CV_64F));
cv::Mat F(3, 3, CV_64F);
//Convert rotation vector into rotation matrix
cv::Rodrigues(rotVec1.at(0), rotMat1);
cv::Rodrigues(rotVec2.at(0), rotMat2);
//Transform prameters of camera 2 into world
rotMat2 = rotMat2.t();
transVec2toWorld = (-rotMat2 * transVec2.at(0));
//Compute R, T to rotate, translate image b into image a
R = rotMat2 * rotMat1; //or --> rotMat2 * rotMat1 ???
T = transVec1.at(0) + transVec2toWorld;
//Compose skew matrix (Format: S =
// [0,-T[3], T[2]; -> 1
S.at<double>(0, 1) = -T.at<double>(2); S.at<double>(0, 2) = T.at<double>(1); //-> 1
// T[3], 0, -T[1]; -> 2
S.at<double>(1, 0) = T.at<double>(2); S.at<double>(1, 2) = -T.at<double>(0); //-> 2
// -T[2], T[1], 0];) -> 3
S.at<double>(2, 0) = -T.at<double>(1); S.at<double>(2, 1) = T.at<double>(0); //-> 3
//Compute fundamental matrix (F = inverse(transpose(camMat2)) * R * S * inverse(camMat1))
return cv::Mat(camMat2.t().inv() * R * S * camMat1.inv());
}
Tue, 13 Dec 2016 11:34:36 -0600http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/Answer by Ice_T02 for <p>Hello,</p>
<p>I try to compute the fundamental matrix given the following camera calibration parameters:</p>
<ul>
<li>camera Matrix 1/2 (camMat1, camMat2)</li>
<li>rotation Vector 1/2 (rotVec1, rotVec2)</li>
<li>translation Vector 1/2 (transVec1, transVec2)</li>
</ul>
<p>According to the following formula the fundamental matrix F is computed by:</p>
<pre><code>F = inverse(transpose(camMat1)) * R * S * inverse(camMat2)
</code></pre>
<p>Anyway, i am quite a bit lost how to compute R and S. I know that R is the rotation matrix which brings image 1 into image 2. Also i know that S is the translation vector to transform image 1 into image 2. My plan would be:</p>
<p>1) Rodrigues both rotation Vectors and substract rotation Matrix 1 from rotation Matrix 2</p>
<pre><code>cv::Mat rotMat1, rotMat2;
cv::Rodrigues(rotVec1[0], rotMat1);
cv::Rodrigues(rotVec2[0], rotMat2);
cv::Mat R = rotMat2 - rotMat1;
</code></pre>
<p>2) Substract translation Vector 1 from translation Vector 2</p>
<pre><code>T = transVec2 - transVec1
</code></pre>
<p>3) Compose S</p>
<pre><code>S = [0,-T[3], T[2]; T[3], 0, -T[1]; -T[2], T[1], 0];
</code></pre>
<p>Would this be correct? Any help on this topic would be appreciated.
I hope this is not offtopic since it is not directly related to OpenCV. </p>
<p><em>Edit: I worked in the solution you provided. I cross checked it with cv::stereoCalib(). Unfortunatly the matrizes does not match. Any suggestions what i did wrong?</em></p>
<pre><code>cv::Mat computeFundMat(cv::Mat camMat1, cv::Mat camMat2, vector<cv::Mat> rotVec1,
vector<cv::Mat> rotVec2, vector<cv::Mat> transVec1, vector<cv::Mat> transVec2)
{
cv::Mat rotMat1(3, 3, CV_64F), rotMat2(3, 3, CV_64F);
cv::Mat transVec2toWorld(3, 1, CV_64F);
cv::Mat R(3, 3, CV_64F), T(3, 1, CV_64F), S(cv::Mat::zeros(3, 3, CV_64F));
cv::Mat F(3, 3, CV_64F);
//Convert rotation vector into rotation matrix
cv::Rodrigues(rotVec1.at(0), rotMat1);
cv::Rodrigues(rotVec2.at(0), rotMat2);
//Transform prameters of camera 2 into world
rotMat2 = rotMat2.t();
transVec2toWorld = (-rotMat2 * transVec2.at(0));
//Compute R, T to rotate, translate image b into image a
R = rotMat2 * rotMat1; //or --> rotMat2 * rotMat1 ???
T = transVec1.at(0) + transVec2toWorld;
//Compose skew matrix (Format: S =
// [0,-T[3], T[2]; -> 1
S.at<double>(0, 1) = -T.at<double>(2); S.at<double>(0, 2) = T.at<double>(1); //-> 1
// T[3], 0, -T[1]; -> 2
S.at<double>(1, 0) = T.at<double>(2); S.at<double>(1, 2) = -T.at<double>(0); //-> 2
// -T[2], T[1], 0];) -> 3
S.at<double>(2, 0) = -T.at<double>(1); S.at<double>(2, 1) = T.at<double>(0); //-> 3
//Compute fundamental matrix (F = inverse(transpose(camMat2)) * R * S * inverse(camMat1))
return cv::Mat(camMat2.t().inv() * R * S * camMat1.inv());
}
</code></pre>
http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?answer=160129#post-id-160129I had to suspend this part for a while but i came up with the following solution:
1) Compute Projection Matrix for each camera
cv::Mat rotMat(3, 3, CV_64F), rotTransMat(3, 4, CV_64F);
//Convert rotation vector into rotation matrix
cv::Rodrigues(rotVec.at(0), rotMat);
//Append translation vector to rotation matrix
cv::hconcat(rotMat, transVec.at(0), rotTransMat);
//Compute projection matrix by multiplying intrinsic parameter
//matrix (A) with 3 x 4 rotation and translation pose matrix (RT).
//Formula: Projection Matrix = A * RT;
return (camMat * rotTransMat);
2) Out of the 2 projection matrizes we can compute the fundamental matrix (P1/P2 are the earlier calculated projection marizes)
cv::Mat_<double> X[3];
cv::vconcat(P1.row(1), P1.row(2), X[0]);
cv::vconcat(P1.row(2), P1.row(0), X[1]);
cv::vconcat(P1.row(0), P1.row(1), X[2]);
cv::Mat_<double> Y[3];
cv::vconcat(P2.row(1), P2.row(2), Y[0]);
cv::vconcat(P2.row(2), P2.row(0), Y[1]);
cv::vconcat(P2.row(0), P2.row(1), Y[2]);
cv::Mat_<double> XY;
for (unsigned int i = 0; i < 3; ++i)
{
for (unsigned int j = 0; j < 3; ++j)
{
cv::vconcat(X[j], Y[i], XY);
F.at<double>(i, j) = cv::determinant(XY);
}
}
Solution is taken from [here](http://answers.opencv.org/question/131017/how-do-i-compute-the-fundamental-matrix-from-2-projection-matrices/). Maybe this is helpfull for ppl coming from google.Wed, 14 Jun 2017 03:51:42 -0500http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?answer=160129#post-id-160129Answer by Tetragramm for <p>Hello,</p>
<p>I try to compute the fundamental matrix given the following camera calibration parameters:</p>
<ul>
<li>camera Matrix 1/2 (camMat1, camMat2)</li>
<li>rotation Vector 1/2 (rotVec1, rotVec2)</li>
<li>translation Vector 1/2 (transVec1, transVec2)</li>
</ul>
<p>According to the following formula the fundamental matrix F is computed by:</p>
<pre><code>F = inverse(transpose(camMat1)) * R * S * inverse(camMat2)
</code></pre>
<p>Anyway, i am quite a bit lost how to compute R and S. I know that R is the rotation matrix which brings image 1 into image 2. Also i know that S is the translation vector to transform image 1 into image 2. My plan would be:</p>
<p>1) Rodrigues both rotation Vectors and substract rotation Matrix 1 from rotation Matrix 2</p>
<pre><code>cv::Mat rotMat1, rotMat2;
cv::Rodrigues(rotVec1[0], rotMat1);
cv::Rodrigues(rotVec2[0], rotMat2);
cv::Mat R = rotMat2 - rotMat1;
</code></pre>
<p>2) Substract translation Vector 1 from translation Vector 2</p>
<pre><code>T = transVec2 - transVec1
</code></pre>
<p>3) Compose S</p>
<pre><code>S = [0,-T[3], T[2]; T[3], 0, -T[1]; -T[2], T[1], 0];
</code></pre>
<p>Would this be correct? Any help on this topic would be appreciated.
I hope this is not offtopic since it is not directly related to OpenCV. </p>
<p><em>Edit: I worked in the solution you provided. I cross checked it with cv::stereoCalib(). Unfortunatly the matrizes does not match. Any suggestions what i did wrong?</em></p>
<pre><code>cv::Mat computeFundMat(cv::Mat camMat1, cv::Mat camMat2, vector<cv::Mat> rotVec1,
vector<cv::Mat> rotVec2, vector<cv::Mat> transVec1, vector<cv::Mat> transVec2)
{
cv::Mat rotMat1(3, 3, CV_64F), rotMat2(3, 3, CV_64F);
cv::Mat transVec2toWorld(3, 1, CV_64F);
cv::Mat R(3, 3, CV_64F), T(3, 1, CV_64F), S(cv::Mat::zeros(3, 3, CV_64F));
cv::Mat F(3, 3, CV_64F);
//Convert rotation vector into rotation matrix
cv::Rodrigues(rotVec1.at(0), rotMat1);
cv::Rodrigues(rotVec2.at(0), rotMat2);
//Transform prameters of camera 2 into world
rotMat2 = rotMat2.t();
transVec2toWorld = (-rotMat2 * transVec2.at(0));
//Compute R, T to rotate, translate image b into image a
R = rotMat2 * rotMat1; //or --> rotMat2 * rotMat1 ???
T = transVec1.at(0) + transVec2toWorld;
//Compose skew matrix (Format: S =
// [0,-T[3], T[2]; -> 1
S.at<double>(0, 1) = -T.at<double>(2); S.at<double>(0, 2) = T.at<double>(1); //-> 1
// T[3], 0, -T[1]; -> 2
S.at<double>(1, 0) = T.at<double>(2); S.at<double>(1, 2) = -T.at<double>(0); //-> 2
// -T[2], T[1], 0];) -> 3
S.at<double>(2, 0) = -T.at<double>(1); S.at<double>(2, 1) = T.at<double>(0); //-> 3
//Compute fundamental matrix (F = inverse(transpose(camMat2)) * R * S * inverse(camMat1))
return cv::Mat(camMat2.t().inv() * R * S * camMat1.inv());
}
</code></pre>
http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?answer=118758#post-id-118758 Not quite. First, rotation matrices don't work that way, and second, coordinate systems.
There is more information on rotation matrices [HERE](https://en.wikipedia.org/wiki/Rotation_matrix#In_three_dimensions), but to start there are three things you need to know.
1. You can invert a rotation matrix by transposing it (rotMat.t())
2. You combine rotation matrices by matrix multiplying them (the * operator, ex rotMat1*rotMat2)
3. The rotations and translations you get from calibration transform world points to camera relative points.
Bearing number 3 in mind, if you combine the two rotation matrices together directly, you would be transforming from world to camera1, then from world to camera2. This is wrong because after the first, the point isn't a world point any more.
You pick one of the cameras to be the origin (use camera1) and change the rotation and translation of the other to be the camera relative to the world. If you want to really understand what's going on, you should derive this for yourself, but that's always easier if you know what you're going for, so here's code to reverse the rotation and translation vector and get the transformation from camera to world.
Rodrigues(rvec, cameraRotation); //3x1 to 3x3
cameraRotation = cameraRotation.t(); //Invert the rotation matrix
cameraTranslation = (-cameraRotation * tvec); //Appropriately transform the translation vector
So, now you have camera2->world and world->camera1. You combine them by multiplying the rotation matrices (I always forget which order, I think it's rotMat1*rotMat2, but try both to be sure). And combing translation is as easy as transVec1+transVec2.
Now you have an R and T from camera2->camera1, and you have what you need.
Or almost. Where did you get your original rvec and tvecs? If you used camera calibration, did you use a static scene? Was the calibration target in exactly the same physical location for both calculations? Since world points are defined as places on the calibration target, if you moved the target, you moved your world points, and you actually have two, unknown world frames. If necessary, take a new picture of an unmoving scene and use solvePnP to get your rvecs and tvecs.Tue, 13 Dec 2016 18:30:13 -0600http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?answer=118758#post-id-118758Comment by Tetragramm for <p>Not quite. First, rotation matrices don't work that way, and second, coordinate systems.</p>
<p>There is more information on rotation matrices <a href="https://en.wikipedia.org/wiki/Rotation_matrix#In_three_dimensions">HERE</a>, but to start there are three things you need to know.</p>
<ol>
<li>You can invert a rotation matrix by transposing it (rotMat.t())</li>
<li>You combine rotation matrices by matrix multiplying them (the * operator, ex rotMat1*rotMat2)</li>
<li>The rotations and translations you get from calibration transform world points to camera relative points.</li>
</ol>
<p>Bearing number 3 in mind, if you combine the two rotation matrices together directly, you would be transforming from world to camera1, then from world to camera2. This is wrong because after the first, the point isn't a world point any more. </p>
<p>You pick one of the cameras to be the origin (use camera1) and change the rotation and translation of the other to be the camera relative to the world. If you want to really understand what's going on, you should derive this for yourself, but that's always easier if you know what you're going for, so here's code to reverse the rotation and translation vector and get the transformation from camera to world.</p>
<pre><code>Rodrigues(rvec, cameraRotation); //3x1 to 3x3
cameraRotation = cameraRotation.t(); //Invert the rotation matrix
cameraTranslation = (-cameraRotation * tvec); //Appropriately transform the translation vector
</code></pre>
<p>So, now you have camera2->world and world->camera1. You combine them by multiplying the rotation matrices (I always forget which order, I think it's rotMat1*rotMat2, but try both to be sure). And combing translation is as easy as transVec1+transVec2.</p>
<p>Now you have an R and T from camera2->camera1, and you have what you need.</p>
<p>Or almost. Where did you get your original rvec and tvecs? If you used camera calibration, did you use a static scene? Was the calibration target in exactly the same physical location for both calculations? Since world points are defined as places on the calibration target, if you moved the target, you moved your world points, and you actually have two, unknown world frames. If necessary, take a new picture of an unmoving scene and use solvePnP to get your rvecs and tvecs.</p>
http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?comment=119230#post-id-119230To be honest, I have no idea if your formula for the fundamental matrix is correct. If you're just looking to use it, and it seems you are, find it with the function [findFundamentalMat](http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga30ccb52f4e726daa039fd5cb5bf0822b).
Take an image of a static scene (say a chessboard) and find the corners in both images. That function will give you the fundamental matrix. I don't know if it will work at 90 degrees though. Without depth, you can only be sure if they match in the vertical direction, not left/right in either camera.Thu, 15 Dec 2016 20:01:52 -0600http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?comment=119230#post-id-119230Comment by Ice_T02 for <p>Not quite. First, rotation matrices don't work that way, and second, coordinate systems.</p>
<p>There is more information on rotation matrices <a href="https://en.wikipedia.org/wiki/Rotation_matrix#In_three_dimensions">HERE</a>, but to start there are three things you need to know.</p>
<ol>
<li>You can invert a rotation matrix by transposing it (rotMat.t())</li>
<li>You combine rotation matrices by matrix multiplying them (the * operator, ex rotMat1*rotMat2)</li>
<li>The rotations and translations you get from calibration transform world points to camera relative points.</li>
</ol>
<p>Bearing number 3 in mind, if you combine the two rotation matrices together directly, you would be transforming from world to camera1, then from world to camera2. This is wrong because after the first, the point isn't a world point any more. </p>
<p>You pick one of the cameras to be the origin (use camera1) and change the rotation and translation of the other to be the camera relative to the world. If you want to really understand what's going on, you should derive this for yourself, but that's always easier if you know what you're going for, so here's code to reverse the rotation and translation vector and get the transformation from camera to world.</p>
<pre><code>Rodrigues(rvec, cameraRotation); //3x1 to 3x3
cameraRotation = cameraRotation.t(); //Invert the rotation matrix
cameraTranslation = (-cameraRotation * tvec); //Appropriately transform the translation vector
</code></pre>
<p>So, now you have camera2->world and world->camera1. You combine them by multiplying the rotation matrices (I always forget which order, I think it's rotMat1*rotMat2, but try both to be sure). And combing translation is as easy as transVec1+transVec2.</p>
<p>Now you have an R and T from camera2->camera1, and you have what you need.</p>
<p>Or almost. Where did you get your original rvec and tvecs? If you used camera calibration, did you use a static scene? Was the calibration target in exactly the same physical location for both calculations? Since world points are defined as places on the calibration target, if you moved the target, you moved your world points, and you actually have two, unknown world frames. If necessary, take a new picture of an unmoving scene and use solvePnP to get your rvecs and tvecs.</p>
http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?comment=119135#post-id-119135Thank you for providing a solution i am currcently investigating. I use OpenCV camera calibration (single camera calibration). Both cameras are prependicular to another and remain static during calibration. Only the grid will be moved in the scene.
Anyway to give you a bigger picture of what i am trying to achieve. I have written a circle/ellipse detection algorithm to find 2 spheres in the camera images (2 per Image). Sometimes it may occur that i have a false positive in either image 1 or 2. To elimnate these i tought about using epipolar geometry and check if there is a circle on the same location in the corresponding image. I am currently not sure if this approach is correct or can be used.
EDIT: see original postThu, 15 Dec 2016 04:18:36 -0600http://answers.opencv.org/question/118671/compute-fundamental-matrix-from-camera-calibration/?comment=119135#post-id-119135