OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Mon, 11 Mar 2019 13:05:15 -0500Could the Real Time pose estimation tutorial be used for a custom shape object ?http://answers.opencv.org/question/210125/could-the-real-time-pose-estimation-tutorial-be-used-for-a-custom-shape-object/ The tutorial is here: https://docs.opencv.org/3.0-beta/doc/tutorials/calib3d/real_time_pose/real_time_pose.html
and it says: "The application starts up loading the 3D textured model in YAML file format". So I could use any object shape I could given I have the data for it?
ThankstrttrtMon, 11 Mar 2019 13:05:15 -0500http://answers.opencv.org/question/210125/Aruco Markers point position estimationhttp://answers.opencv.org/question/189278/aruco-markers-point-position-estimation/I need to detect and track an area of fixed size (let's say 1mt by 1mt).
I am using 4 sets of 3 markers (each one for a different corner) so that by reading the marker Id I can know which corner it belongs to. Each set of marker has an L disposition (oriented to match the corner shape).
If I can detect all the markers I am able to successfully identify the target area. But, if one or more markers are not detected (e.g. they are outside the frame) I would like to use the others to estimate the missing points.
Unfortuately pose estimation is good locally around the marker but gives very poor results if i try to estimate the position of farther points.
The code I am using at the moment is something like this
Marker m; //filled, imagine this is the bottom left marker and I want the top right one
CameraParameters camParams; //filled
Mat objPoint(1,3,CV_32FC1);
objPoint.at<float>(0,0) = 1.0f;
objPoint.at<float>(0,1) = 1.0f;
objPoint.at<float>(0,2) = 0.0f;
// From the center of the marker I want to move 1mt along x axis and 1mt along y axis
vector<Point2f> imagePoints;
projectPoints(objectPoints, m.Rvec, m.Tvec, camParams.CameraMatrix, camParams.Distorsion, imagePoints);
Point2f estimate = imagePoints[0];
At this point I expect estimate to be the point I wanted expressed in screen pixels. But the result is often very bad. I also tried using PoseTrakcers and "homemade tracking" by saving the previous positions of the markers and then by trying to predict the next.
Am I doing something wrong? Is there a better way?
I am using OpenCV 3.4.1 and Aruco 3.0.6.
Thanks in advance.
BobazThu, 12 Apr 2018 10:03:06 -0500http://answers.opencv.org/question/189278/cv::gpu::solvePnPRansachttp://answers.opencv.org/question/167076/cvgpusolvepnpransac/ I am using opencv 3.1.0 and I cannot find the gpu-based solvePnPRansac method. It is my understanding that the gpu::* methods and types were all moved somewhere else. Where can i find the equivalent of gpu::SolvePnpRansac?fab_cutMon, 17 Jul 2017 04:54:03 -0500http://answers.opencv.org/question/167076/Error in gauss-newton implementation for pose optimizationhttp://answers.opencv.org/question/147038/error-in-gauss-newton-implementation-for-pose-optimization/ Hello. I’m using a modified version of a gauss-newton method to refine a pose estimate using OpenCV. The unmodified code can be found here: [http://people.rennes.inria.fr/Eric.Marchand/pose-estimation/tutorial-pose-gauss-newton-opencv.html][1]
The details of this approach are outlined in the corresponding paper:
> Marchand, Eric, Hideaki Uchiyama, and Fabien Spindler. "Pose
> estimation for augmented reality: a hands-on survey." IEEE
> transactions on visualization and computer graphics 22.12 (2016):
> 2633-2651.
A PDF can be found here: [https://hal.inria.fr/hal-01246370/document][2]
The part that is relevant (Pages 4 and 5) are screencapped below:
[![Gauss-Newton Minimization][3]][3]
Here is what I have done. First, I’ve (hopefully) “corrected” some errors: (a) `dt` and `dR` can be passed by reference to `exponential_map()` (even though `cv::Mat` is essentially a pointer). (b) The last entry of each 2x6 Jacobian matrix, `J.at<double>(i*2+1,5)`, was `-x[i].y` but should be `-x[i].x`. (c) I’ve also tried using a different formula for the projection. Specifically, one that includes the focal length and principal point:
xq.at<double>(i*2,0) = cx + fx * cX.at<double>(0,0) / cX.at<double>(2,0);
xq.at<double>(i*2+1,0) = cy + fy * cX.at<double>(1,0) / cX.at<double>(2,0);
Here is the relevant code I am using, in its entirety (control starts at optimizePose3()):
void exponential_map(const cv::Mat &v, cv::Mat &dt, cv::Mat &dR)
{
double vx = v.at<double>(0,0);
double vy = v.at<double>(1,0);
double vz = v.at<double>(2,0);
double vtux = v.at<double>(3,0);
double vtuy = v.at<double>(4,0);
double vtuz = v.at<double>(5,0);
cv::Mat tu = (cv::Mat_<double>(3,1) << vtux, vtuy, vtuz); // theta u
cv::Rodrigues(tu, dR);
double theta = sqrt(tu.dot(tu));
double sinc = (fabs(theta) < 1.0e-8) ? 1.0 : sin(theta) / theta;
double mcosc = (fabs(theta) < 2.5e-4) ? 0.5 : (1.-cos(theta)) / theta / theta;
double msinc = (fabs(theta) < 2.5e-4) ? (1./6.) : (1.-sin(theta)/theta) / theta / theta;
dt.at<double>(0,0) = vx*(sinc + vtux*vtux*msinc)
+ vy*(vtux*vtuy*msinc - vtuz*mcosc)
+ vz*(vtux*vtuz*msinc + vtuy*mcosc);
dt.at<double>(1,0) = vx*(vtux*vtuy*msinc + vtuz*mcosc)
+ vy*(sinc + vtuy*vtuy*msinc)
+ vz*(vtuy*vtuz*msinc - vtux*mcosc);
dt.at<double>(2,0) = vx*(vtux*vtuz*msinc - vtuy*mcosc)
+ vy*(vtuy*vtuz*msinc + vtux*mcosc)
+ vz*(sinc + vtuz*vtuz*msinc);
}
void optimizePose3(const PoseEstimation &pose,
std::vector<FeatureMatch> &feature_matches,
PoseEstimation &optimized_pose) {
//Set camera parameters
double fx = camera_matrix.at<double>(0, 0); //Focal length
double fy = camera_matrix.at<double>(1, 1);
double cx = camera_matrix.at<double>(0, 2); //Principal point
double cy = camera_matrix.at<double>(1, 2);
auto inlier_matches = getInliers(pose, feature_matches);
std::vector<cv::Point3d> wX;
std::vector<cv::Point2d> x;
const unsigned int npoints = inlier_matches.size();
cv::Mat J(2*npoints, 6, CV_64F);
double lambda = 0.25;
cv::Mat xq(npoints*2, 1, CV_64F);
cv::Mat xn(npoints*2, 1, CV_64F);
double residual=0, residual_prev;
cv::Mat Jp;
for(auto i = 0u; i < npoints; i++) {
//Model points
const cv::Point2d &M = inlier_matches[i].model_point();
wX.emplace_back(M.x, M.y, 0.0);
//Imaged points
const cv::Point2d &I = inlier_matches[i].image_point();
xn.at<double>(i*2,0) = I.x; // x
xn.at<double>(i*2+1,0) = I.y; // y
x.push_back(I);
}
//Initial estimation
cv::Mat cRw = pose.rotation_matrix;
cv::Mat ctw = pose.translation_vector;
int nIters = 0;
// Iterative Gauss-Newton minimization loop
do {
for (auto i = 0u; i < npoints; i++) {
cv::Mat cX = cRw * cv::Mat(wX[i]) + ctw; // Update cX, cY, cZ
// Update x(q)
//xq.at<double>(i*2,0) = cX.at<double>(0,0) / cX.at<double>(2,0); // x(q) = cX/cZ
//xq.at<double>(i*2+1,0) = cX.at<double>(1,0) / cX.at<double>(2,0); // y(q) = cY/cZ
xq.at<double>(i*2,0) = cx + fx * cX.at<double>(0,0) / cX.at<double>(2,0);
xq.at<double>(i*2+1,0) = cy + fy * cX.at<double>(1,0) / cX.at<double>(2,0);
// Update J using equation (11)
J.at<double>(i*2,0) = -1 / cX.at<double>(2,0); // -1/cZ
J.at<double>(i*2,1) = 0;
J.at<double>(i*2,2) = x[i].x / cX.at<double>(2,0); // x/cZ
J.at<double>(i*2,3) = x[i].x * x[i].y; // xy
J.at<double>(i*2,4) = -(1 + x[i].x * x[i].x); // -(1+x^2)
J.at<double>(i*2,5) = x[i].y; // y
J.at<double>(i*2+1,0) = 0;
J.at<double>(i*2+1,1) = -1 / cX.at<double>(2,0); // -1/cZ
J.at<double>(i*2+1,2) = x[i].y / cX.at<double>(2,0); // y/cZ
J.at<double>(i*2+1,3) = 1 + x[i].y * x[i].y; // 1+y^2
J.at<double>(i*2+1,4) = -x[i].x * x[i].y; // -xy
J.at<double>(i*2+1,5) = -x[i].x; // -x
}
cv::Mat e_q = xq - xn; // Equation (7)
cv::Mat Jp = J.inv(cv::DECOMP_SVD); // Compute pseudo inverse of the Jacobian
cv::Mat dq = -lambda * Jp * e_q; // Equation (10)
cv::Mat dctw(3, 1, CV_64F), dcRw(3, 3, CV_64F);
exponential_map(dq, dctw, dcRw);
cRw = dcRw.t() * cRw; // Update the pose
ctw = dcRw.t() * (ctw - dctw);
residual_prev = residual; // Memorize previous residual
residual = e_q.dot(e_q); // Compute the actual residual
std::cout << "residual_prev: " << residual_prev << std::endl;
std::cout << "residual: " << residual << std::endl << std::endl;
nIters++;
} while (fabs(residual - residual_prev) > 0);
//} while (nIters < 30);
optimized_pose.rotation_matrix = cRw;
optimized_pose.translation_vector = ctw;
cv::Rodrigues(optimized_pose.rotation_matrix, optimized_pose.rotation_vector);
}
Even when I use the functions as given, it does not produce the correct results. My initial pose estimate is very close to optimal, but when I try run the program, the method takes a very long time to converge - and when it does, the results are very wrong. I’m not sure what could be wrong and I’m out of ideas. I’m confident my inliers are actually inliers (they were chosen using an M-estimator). I’ve compared the results from exponential map with those from other implementations, and they seem to agree.
So, where is the error in this gauss-newton implementation for pose optimization? I’ve tried to make things as easy as possible for anyone willing to lend a hand. Let me know if there is anymore information I can provide. Any help would be greatly appreciated. Thanks.
[1]: http://people.rennes.inria.fr/Eric.Marchand/pose-estimation/tutorial-pose-gauss-newton-opencv.html
[2]: https://hal.inria.fr/hal-01246370/document
[3]: https://i.stack.imgur.com/xPGia.pngsocraticMon, 08 May 2017 15:44:38 -0500http://answers.opencv.org/question/147038/Entering >5 points into "findEssentialMat"?http://answers.opencv.org/question/89595/entering-5-points-into-findessentialmat/Hi,
I'm trying to figure out the relative pose (R, t matrices) of two cameras using two synchronized videos of the same scene viewed from different positions. As input I've selected points of the *same* object in corresponding frames of both videos, and findEssentialMat needs at least 5.
My question is: what happens if I feed in more than 5 problems? Ideally I'd like to feed in many more points than 5 points and get a more accurate estimate than just 5 points would give. That's what I hope. Otherwise I'd consider finding R,t from many pairs of 5 points of points, then taking the median or mean as the best estimate.
Thanks.gandalfsaxeMon, 07 Mar 2016 08:36:42 -0600http://answers.opencv.org/question/89595/relative pose between two imageshttp://answers.opencv.org/question/75562/relative-pose-between-two-images/Is it possible to get a relative pose between two images?
When I got E => (R, t) and K shouldn't it be possible to calculate the location of a point in the second image, found in the first image.
I know you need to know the scale factor for full reconstruction but do I really need it two if I just want a relative pose.
And lets say when there is only a rotation then a least the scaling factor should not be interesting?!
but why does it then be in this formula with R combined?
s2* m'2 = R * s1 * m'1 + tMJFri, 06 Nov 2015 10:27:51 -0600http://answers.opencv.org/question/75562/Units of Rotation and translation from Essential Matrixhttp://answers.opencv.org/question/66839/units-of-rotation-and-translation-from-essential-matrix/ I obtained my Rotation and translation matrices using the SVD of E. I used the following code to do so :
SVD svd(E);
Matx33d W(0, -1, 0, 1, 0, 0, 0, 0, 1);
Mat_<double> R = svd.u * Mat(W) * svd.vt;
Mat_<double> t = svd.u.col(2);
Matx34d P1(R(0, 0), R(0, 1), R(0, 2), t(0), R(1, 0), R(1, 1), R(1, 2), t(1), R(2, 0), R(2, 1), R(2, 2), t(2));
I want to know what are the units of R and t ? When I calibrated the cameras I got the baseline distance in **cm** , So is the translation vector units also in **cm** ? Also what are the units of R?
P.S : I have read the following question on [Stackoverflow](https://stackoverflow.com/questions/16856177/what-will-be-the-translation-vector-unit-from-essential-matrix/16870447#16870447) , but it had actually confused me as no answer was accepted. Also I wanted to know if the translation vector I got is it in the world frame?(Is it the real world distance moved from the initial position)
EDIT 1:
I have also observed that the values are normalized for the translation vector i.e x^2+y^2+z^2 is nearly 1. So how do I get the actual vector?
EDIT 2: I have read the following question in [Stackoverflow](https://stackoverflow.com/questions/3678317/t-and-r-estimation-from-essential-matrix?rq=1) and I think I will be implementing it. Don't know if there is any better way.Sai KondaveetiWed, 22 Jul 2015 22:11:34 -0500http://answers.opencv.org/question/66839/