Hello,
I have a process which makes a sparse point cloud from a set of 2D images, basically SfM. The output of this algorithm contains K, R and T matrices for triangulation of 3D points and reprojecting them back into the 2D image.
I now want to produce an orthomosaic as well, so I started to use the stitcher, which uses its own feature detector and bundle adjuster and using a transverse mercator warper I already got quite a decent result.
I would now like to see if I can improve the results by applying the R|T matrix from the sparse point cloud on the stitcher cameras instead. The R|t matrices of that camera isn't the same though, as mentioned in this post:
http://answers.opencv.org/question/26821/a-question-about-relation-of-k-r-t-h/
My question is... how does one convert/map an R|T matrix from a 3D SfM process into a R|t matrix used in the stitcher in opencv code, or is that very involved? The rotation in 3D space (x/y/z) of this sparse point cloud is arbitrary.
The motion estimator.cpp code contains this fragment:
Mat R = K_from.inv() * pairwise_matches[pair_idx].H.inv() * K_to;
cameras[edge.to].R = cameras[edge.from].R * R;
which effectively assigns a rotation to a camera with respect to the projection plane. H defines the rotation between two images, so the R for the new camera is a concatenation of the rotation so far concatenated with the new rotation defined by the homography.
The r and t here are from the P=K[R|T] camera model, which is accurately determined. I'm eventually looking for the R and t matrices that are used in the stitcher, which seem different from the R|T above.
25-04-2014 HYPOTHESIS:
I figured out a different way for my problem, but would still like to see this question answered. From Robert Collins, CSE486 Penn state, "Projection of Points on Planar Surface":
[x] [f 0 0 0] [r11 r12 r13 tx] [p]
[y] ~ [0 f 0 0] [r21 r22 r23 ty] [q]
[1] [0 0 1 0] [r31 r32 r33 tz] [0]
[ 0 0 0 1 ]
[1] [1]
[x] [f 0 0 0] [r11 r12 tx] [p]
[y] ~ [0 f 0 0] [r21 r22 ty] [q]
[1] [0 0 1 0] [r31 r32 tz] [1]
[ 0 0 1
] ]
[x] [f 0 0] [r11 r12 tx] [p]
[y] ~ [0 f 0] [r21 r22 ty] [q]
[1] [0 0 1] [r31 r32 tz]
[1] [1]
[x] [fr11 fr12 ftx] [p]
[y] ~ [fr21 fr22 fty] [q]
[1] [r31 r32 tz]
[1] [1]
[x] [h11 h12 h13] [p]
[y] ~ [h21 h22 h23] [q]
[1] [h31 h32 h33]
[1][1]
Which apparently defines the homography.
Which would make this in code:
Mat_<double> H = Mat::eye(3, 3, CV_64F);
H.at<double>(0,0) = f * R.at<double>(0,0);
H.at<double>(0,1) = f * R.at<double>(0,1);
H.at<double>(1,0) = f * R.at<double>(1,0);
H.at<double>(1,1) = f * R.at<double>(1,1);
H.at<double>(2,0) = R.at<double>(2,0);
H.at<double>(2,1) = R.at<double>(2,1);
H.at<double>(0,2) = f * T[0];
H.at<double>(1,2) = f * T[1];
H.at<double>(2,2) =
T[2];T[2];
Can I now just calculate R through:
through:
Mat R = K_from.inv() * pairwise_matches[pair_idx].H.inv() * K_to;
?
