OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Fri, 18 Dec 2020 07:38:58 -0600Problem with undistortPoints() function in pose estimation of imagehttp://answers.opencv.org/question/239443/problem-with-undistortpoints-function-in-pose-estimation-of-image/ I have written about my task [here](https://answers.opencv.org/question/238792/problem-with-building-pose-mat-from-rotation-and-translation-matrices/). I have
a set of images with known pose which were used for scene reconstruction and some query image from the same space without pose. I need to calculate the pose of the query image. I solved this problem using essential matrix. Here is a code
Mat E = findEssentialMat(pts1, pts2, focal, pp, FM_RANSAC, F_DIST, F_CONF, mask);
// Read pose for view image
Mat R, t; //, mask;
recoverPose(E, pts1, pts2, R, t, focal, pp, mask);
The only problem is that OpenCv documentation [states](https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga13f7e34de8fa516a686a56af1196247f) that the findEssentialMat function assumes that points1 and points2 are feature points from cameras with the same camera intrinsic matrix. That's not a case for us - images of scene and query image can be captured by cameras with different intrinsics.
I suppose to use this undistortPoints() function. According to [documentation](https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga55c716492470bfe86b0ee9bf3a1f0f7e) the undistortPoints() function takes two important parameters distCoeffs and cameraMatrix.
Both images of the scene and query image have calibration parameters associated (fx, fy, cx, cy).
I obtain cameraMatrix parameter this way:
Mat K_v = (Mat_<double>(3, 3) <<
fx, 0, cx,
0, fy, cy,
0, 0, 1, CV_64F);
Is this correct? Moreover I need to get somewhere distCoeffs. So how I can obtain this distortion coefficients for images of the scene and query image? Probably I should solve it another way?sigmoid90Fri, 18 Dec 2020 07:38:58 -0600http://answers.opencv.org/question/239443/Problem with building pose Mat from rotation and translation matriceshttp://answers.opencv.org/question/238792/problem-with-building-pose-mat-from-rotation-and-translation-matrices/I have two images captured in the same space (scene), one with known pose. I need to calculate the pose of the second (query) image. I have obtained the relative camera pose using essential matrix. Now I am doing calculation of camera pose through the matrix multiplication ([here](https://answers.opencv.org/question/31421/opencv-3-essentialmatrix-and-recoverpose/) is a formula).
I try to build the 4x4 pose Mat from rotation and translation matrices. My code is following
Pose bestPose = poses[best_view_index];
Mat cameraMotionMat = bestPose.buildPoseMat();
cout << "cameraMotionMat: " << cameraMotionMat.rows << ", " << cameraMotionMat.cols << endl;
float row_a[4] = {0.0, 0.0, 0.0, 1.0};
Mat row = Mat::zeros(1, 4, CV_64F);
cout << row.type() << endl;
cameraMotionMat.push_back(row);
// cameraMotionMat.at<float>(3, 3) = 1.0;
Earlier in code. Fora each view image:
Mat E = findEssentialMat(pts1, pts2, focal, pp, FM_RANSAC, F_DIST, F_CONF, mask);
// Read pose for view image
Mat R, t; //, mask;
recoverPose(E, pts1, pts2, R, t, focal, pp, mask);
Pose pose (R, t);
poses.push_back(pose);
Initially method bestPose.buildPoseMat() returns Mat of size (3, 4). I need to extend the Mat to size (4, 4) with row [0.0, 0.0, 0.0, 1.0] (zeros vector with 1 on the last position). Strangely I get following output when print out the resultant matrix
> [0.9107258520121255,
> 0.4129580377861768, 0.006639390377046724, 0.9039011699443721;
> 0.4129661348384583, -0.9107463665340377, 0.0001652925667582038, -0.4277340727282191;
> 0.006115059555925467, 0.002591307168000504, -0.9999779453436902, 0.002497598952195387;
> 0, 0.0078125, 0, 0]
Last row does not look like it should be: [0, 0.0078125, 0, 0] rather than [0.0, 0.0, 0.0, 1.0]. Is this implementation correct? What could a problems with this matrix?sigmoid90Sun, 06 Dec 2020 04:51:36 -0600http://answers.opencv.org/question/238792/Estimate camera pose from homography?http://answers.opencv.org/question/234966/estimate-camera-pose-from-homography/Hello, having two non-planar images, I am trying to estimate the pose of the camera using the homography matrix (which normally is used for planar images), the fundamental matrix, and the essential matrix.
In the first case, I am interested in the homography matrix.
So I had to use one of the descriptors exiting on openCV (Sift, Surf, ORB, Akaze), before matching the feature points (using FLANN or BFMatcher), then I calculated the homography matrix with ***findHomographyMat*** using Ransac or other.
once I have the homography matrix, I decompose it using ***decomposeHomographyMat*** which returns 4 possible cases of transformation (Rotation matrix, translation vector, and Normal vector)
in order to confirm that the result is correct, we can recompose the homography matrix using the transformations given by ***decomposeHomographyMat*** using the following formula:
NumberOfResult, self.Rs, self.Ts, self.Ns = cv.decomposeHomographyMat(self.H, np.eye(3))
# /** H = Rs + Ts * Ns.T **/ #
# /** Nomalize H --> H = H / H(2,2) **/ #
for i in range (NumberOfResult):
recomputeH = Rs[i] + np.dot(Ts[i], Ns[i].T)
norm = recomputeH[2][2]
recomputeH = (1 / norm) * recomputeH
# Just for confirmation result
print (np.array_equal(np.around(self.H, decimals=4), np.around(recomputeH, decimals=4)))
now having this transformation, I would like to assign a 3d position to camera 1, and assign it the transformation to have the pose of camera 2, and I don't know how to do that, if anyone has any idea, please answer meRashidLadjThu, 10 Sep 2020 05:48:17 -0500http://answers.opencv.org/question/234966/why invert posehttp://answers.opencv.org/question/232411/why-invert-pose/ I have a pose output from solvepnp(), not a prob. Many sources suggest to take the inverse of the pose. I am having a contextual blindspot here: can someone tell me why I should use the inverted pose. In common sense language how is the inverted coordinate space more accurate/more appropriate.superflySun, 12 Jul 2020 13:15:56 -0500http://answers.opencv.org/question/232411/1D->2D pose estimation - detect 2D camera locationhttp://answers.opencv.org/question/228767/1d-2d-pose-estimation-detect-2d-camera-location/ I believe I am off the track. I need to achieve something like a 1D->2D pose estimation, which I was hoping to be able to reduce to the 3D pose estimation problem, to which widely used solutions exist, such as OpenCVs `solvePnP()` implementations.
What I am trying to solve:
My camera is point _along a surface_, i.e. it doesn't sees the surface from the side and sees everything that is on the surface. That means, I already know the z-coordinate which is `z=0`.
I have 2 cameras pointed at the surface and I want to detect objects on the surface by intersecting their projection lines. But for that, I need to know the 2D location of my cameras with respect to the 2D coordinate system of the surface.
The cameras are fixed, so I should be able to determine each of their location by placing objects on the surface in a known setup, and from the object positions on the image, determine the camera location. Although my camera images are 2D, I think I can treat them to be 1D, because the objects will be all on the same y-axis. (Assuming leveled and aligned camera setup).
By that, I believe that the problem I need to solve is somethingl ike a **1D->2D pose estimation**.
I can't find a geometrical solution to figuring out the 2D location from a known set of 1D points, so I was trying to use `solvePnP()`, feeding it only the dimensions I need. I have calibrated my camera to get the camera matrix and distortion vector, and also applied `solvePnP()` in 3D space - which gave me okay results and showed me that my implementation seems alright.
But when I provide the 1D image points (i.e. points which all have the same y-coordinate), the result is not usable. I was lucky to use once points with slightly different y-coordinates, which actually produced a valid camera location in world coordinates (`z` was near zero). So I think this is exactly a limitation of the PnP solutions, that they need some assymetry in the image points. Using them as 1D points puts them all on the same line, which can't help the solvers.
But now, what can I do? Shouldn't the 1D-2D pose estimation be a more simple to solve problem than the 2D-3D one? Can anyone help me to think of a geometrical solution or guide me to some other way of interpreting the **problem of locating objects on a surface**?
Any feedback, hints, discussions are highly appreciated!
Below am providing some sketches to (hopefully) illustrate the setup and how I am trying to solve it with pose estimation.
The world, looking at the surface from the top, and from the side (as the camera).
- `C`: The camera, with it's measured real world location around (20,-340).
- `o<N>`: object points 1-4. These could be some pins or anything.
- x/y/z: The world coordinate system, how I define it.
- FOV: field of view of the camera
![image description](/upfiles/15864366955970161.jpg)
The image, taken from camera `C`. All objects are roughly at the center/zero line.
![image description](/upfiles/15864369458825849.jpg)noshkyThu, 09 Apr 2020 07:56:43 -0500http://answers.opencv.org/question/228767/PnP pose estimation using undistorted images?http://answers.opencv.org/question/228324/pnp-pose-estimation-using-undistorted-images/Hi, I am getting currently into camera calibration, undistortion and eventually I'm going to attempt pose estimation to detect the position of the camera in the world. There is a long way ahead of me, as I have to learn all about object coordinates and world coordinates and all between.
However, I have a fairly simple question to ask, which I couldn't find clues for in all my readings.
So I got my camera matrix and distance vectors thanks to calibration, and now I'm ready to shoot them into [``solvePnP()``](https://docs.opencv.org/4.2.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d) to know where I am.
In my image I got from the camera, I already recognized the 2D coordinates for the world coordinates which are known to me. **BUT** did I do that on the **raw camera** image? Or did I **undistort** the image before?
That is basically my question. As I can't really follow the maths of what is going behind the scenes, I am hoping to get some answers here. That'd be awesome!
Thanks a lot.noshkyTue, 31 Mar 2020 16:25:07 -0500http://answers.opencv.org/question/228324/Essential Matrix from bearing vectorshttp://answers.opencv.org/question/225334/essential-matrix-from-bearing-vectors/openMVG has an essential matrix solver using bearing vectors (from SIFT 2D features matches from a spherical panorama) - see [robust_essential_spherical.cpp](https://github.com/openMVG/openMVG/tree/master/src/openMVG_Samples/multiview_robust_essential_spherical). It works well for retrieving the pose between 2 panoramas. The findEssentialMatrix in OpenCV doesn't support 3D points as such and providing it 2D points from an equirectangular image finds inliers but doesn't find the essential matrix correctly. Is there a similar path of finding the essential matrix (poses) of equirectangular images that doesn't involve converting to cubic format and using the epipolar geometry methods?seanfloridaSun, 26 Jan 2020 15:17:25 -0600http://answers.opencv.org/question/225334/Camera pose estimation with QR code by Opencvhttp://answers.opencv.org/question/222349/camera-pose-estimation-with-qr-code-by-opencv/ Hello,
I am detecting a QR code with ZBAR library, and I have as information the 4 corners of the QR code, I can't find an answer to my question:
is it possible by using only the position of the QR code (4 corners) to estimate the pose of the camera: if the camera is above, below, on the left or in the right of the QR code, the result of the pose can be for example an angle in degrees.
And thak you.KitnosWed, 27 Nov 2019 10:44:59 -0600http://answers.opencv.org/question/222349/Get angles from pose OpenCV (display a video on top of homography)http://answers.opencv.org/question/216358/get-angles-from-pose-opencv-display-a-video-on-top-of-homography/ I have a simple OpenCV program that tries to find a particular image in the camera and if it finds it, it computes its `homography`. This part is working perfectly. I now want to display a video onto this homography in Javascript / HTML. To do this, I need to compute the angle (x,y,z), as well as the size and position of the of the video to put on the homography.
For the sizing, I figured I could do this by averaging the distances between the 4 points from the `homography` horizontally and vertically / or using the max values. For the position, I was just going to find the average and thus center of the 4 points. Please let me know if this is a good idea on how to do this?
For the angles, this seems a bit more complicated. I found that I could estimate the pose with a [script like this][1]. And then display [it in HTML like this][2]. The only thing is I'm not actually sure how to get the angles in degrees from the pose?
Any help / advice about this would be much appreciated. Please feel free to give code / pseudo in any language of your preference such as C++, Python, JS, etc... My program is written in OpenCV.js but I realize it's a pretty obscure platform so anything will do.
**What my homography stuff looks like**
var objF1 = new cv.Point2fVector();
var sceneF1 = new cv.Point2fVector();
var HF1 = new cv.Mat();
[HF1, objF1, sceneF1] = getHomography(obj, scene); // OBJ + SCENE COME FROM DMATCHVECTOR AFTER RATIO TEST
var x1, x2, x3, x4;
[x1,x2,x3,x4] = getPoints(HF1);
**Some of the helper functions**
// FIND HOMOGRAPHY & FIND MATCHES WITHIN HOMOGRAPHY MASK
function getHomography(obj, scene) {
var filteredObj = new cv.Point2fVector();
var filteredScene = new cv.Point2fVector();
var H = new cv.Mat();
var mask = new cv.Mat();
H = cv.findHomographyEasyMask(obj, scene, cv.FM_RANSAC, 3.0, mask);
for (let i = 0; i < obj.size(); i++) {
if (mask.data[i] == 1) {
filteredObj.push_back(obj.get(i));
filteredScene.push_back(scene.get(i));
}
}
mask.delete();
return [H, filteredObj, filteredScene];
}
// TRANSFORM PERSPECTIVE AND GET POINTS
function getPoints(H) {
var obj_corners_mat = new cv.Mat(4, 1, cv.CV_32FC2);
var scene_corners_mat = new cv.Mat(4, 1, cv.CV_32FC2);
var view = obj_corners_mat.data32F;
view[0] = 0;
view[1] = 0;
view[2] = img1Raw.cols;
view[3] = 0;
view[4] = img1Raw.cols;
view[5] = img1Raw.rows;
view[6] = 0;
view[7] = img1Raw.rows;
cv.perspectiveTransform(obj_corners_mat, scene_corners_mat, H);
var dataAns = scene_corners_mat.data32F;
var x1 = [dataAns[0], dataAns[1]];
var x2 = [dataAns[2], dataAns[3]];
var x3 = [dataAns[4], dataAns[5]];
var x4 = [dataAns[6], dataAns[7]];
obj_corners_mat.delete();
scene_corners_mat.delete();
return [x1, x2, x3, x4];
}
[1]: https://stackoverflow.com/questions/8927771/computing-camera-pose-with-homography-matrix-based-on-4-coplanar-points
[2]: https://stackoverflow.com/questions/37354506/html5-video-perspective-transform
AntersBearTue, 30 Jul 2019 05:07:06 -0500http://answers.opencv.org/question/216358/Camera Pose from Checkerboard Posehttp://answers.opencv.org/question/209890/camera-pose-from-checkerboard-pose/I am using the [code](https://docs.opencv.org/3.4/d7/d53/tutorial_py_pose.html) given by the OpenCV to find tvecs and rvecs of the camera which I calibrated intrinsically. I do get these two vectors, no problem.
![image description](/upfiles/1551862083456697.png)
rvecs: [[-0.73059634]
[-0.02493155]
[ 0.1556929 ]]
tvecs: [[-2.8194188 ]
[ 1.25920043]
[77.38695895]]
However, I am not sure if these are the values for my camera or my checkerboard. The language used in the tutorial isn't totally clear. But I think these are the values for the checkerboard, like, this isn't the pose of the camera, but rather the pose of the board, which is not why people use this code in the first place.
Also, if these are actually the camera parameters, then one must multiply 77 with the size of one checker that he uses. Mine has 11 mm side, so for me the height would correspond to `77 * 1.1`, which is also wrong since I measured the height and it's not that high.
Something is fishy here.
My question is, how could we go to camera extrinsics from these vectors?RakiWed, 06 Mar 2019 02:43:35 -0600http://answers.opencv.org/question/209890/Camera position in world coordinate is not working but object pose in camera co ordinate system is working properlyhttp://answers.opencv.org/question/194724/camera-position-in-world-coordinate-is-not-working-but-object-pose-in-camera-co-ordinate-system-is-working-properly/I am working on the camera (iphone camera) pose estimation for head mount device (Hololens) using LEDs as a marker, using the solvepnp. I have calibrated the camera below is the camera intrinsic parameters
/* approx model*/
double focal_length = image.cols;
Point2d center = cv::Point2d(image.cols/2,image.rows/2);
iphone_camera_matrix = (cv::Mat_<double>(3,3) << focal_length, 0, center.x, 0 , focal_length, center.y, 0, 0, 1); iphone_dist_coeffs = cv::Mat::zeros(4,1,cv::DataType<double>::type);
/* caliberated(usng opencv) model */
iphone_camera_matrix = (cv::Mat_<double>(3,3) <<839.43920487140315, 0, 240, 0, 839.43920487140315, 424, 0, 0, 1);
iphone_dist_coeffs = (cv::Mat_<double>(5,1) <<4.6476561543838640e-02, -2.0580084834071521, 0, 0 ,2.0182662261396342e+01);
usng solvpnp am able to get the proper object pose in camera co-ordinate system below is the code
cv::solvePnP(world_points, image_points, iphone_camera_matrix, iphone_dist_coeffs, rotation_vector, translation_vector, true, SOLVEPNP_ITERATIVE);
the ouput is
rotation_vector :
[-65.41956646885059;
-52.49185328449133;
36.82917796058498]
translation_vector :
[94.1158604375937;
-164.2178023980637;
580.5666657301058]
using this rotation_vector and translation_vector am visualizing pose by projecting the trivector whose points are
points_to_project :
[0, 0, 0;
20, 0, 0;
0, 20, 0;
0, 0, 20]
projectPoints(points_to_project, rotation_vector, translation_vector, iphone_camera_matrix, iphone_dist_coeffs, projected_points);
the output of projectedPoints given as
projected_points :
[376.88803, 185.15131;
383.05768, 195.77643;
406.46454, 175.12997;
372.67371, 155.56181]
which seems correct as shown below
![object pose in camera co-ordinate system](/upfiles/15302624392884724.png)
I try to find the camera pose in world/object coordinate system by using the transformation of rotation_vector and translation_vector given by solvepnp as
cv::Rodrigues(rotation_vector, rotation_matrix);
rot_matrix_wld = rotation_matrix.t();
translation_vec_wld = -rot_matrix_wld * translation_vector;
I used the rot_matrix_wld, translation_vec_wld to visualize the pose (same way as how I visualized the pose of the object in the camera coordinate system as said in the above)
projectPoints(points_to_project, rot_matrix_wld, translation_vec_wld, iphone_camera_matrix, iphone_dist_coeffs, projected_points);
with
points_to_project :
[0, 0, 0;
20, 0, 0;
0, 20, 0;
0, 0, 20]
am getting wrong translation vector (below 2 projected_points are for 2 different image frames of a video)
projected_points :
[-795.11768, -975.85846;
-877.84937, -932.39697;
-868.5517, -1197.4443;
projected_points :
[589.42999, 3019.0732;
590.64789, 2665.5835;
479.49728, 2154.8057;
187.78407, 3333.3054]
-593.41058, -851.74432]
I have used the approx camera model and calibrated camera model both are giving the wrong translation vector.
I have gone through the link [here](https://stackoverflow.com/questions/47723638/output-from-solvepnp-doesnt-match-projectpoints) and verified my calibration procedure, I did it correctly.
I am not sure where am doing wrong can anyone please help me with this.
thanks in advance.
slvFri, 29 Jun 2018 03:58:36 -0500http://answers.opencv.org/question/194724/How to interpret the rotation matrices of the homographybasedestimator?http://answers.opencv.org/question/189502/how-to-interpret-the-rotation-matrices-of-the-homographybasedestimator/Good morning,
I have question regarding OpenCV's homographyBasedEstimator. While reading the source code I found out that it determines a set of transformations, which relate every input image to a reference image by multiplying the homographies from the pairwise matching step. Before the multiplication each homography is normalized
(R = K_from.inv() * pairwise_matches[pair_idx].H.inv() * K_to).
What I do not fully understand is how to interpret the rotations returned by this method.
1. Are they just homographies which relate each image to the reference image?
2. Or can they be interpreted as the camera's pose, i.e., its external orientation, because they are used as initial guess for the subsequent bundle adjustment?
Can you guys please help me?
Kind regards
ConundraahConundraahTue, 17 Apr 2018 02:50:00 -0500http://answers.opencv.org/question/189502/composeRT input/output questionhttp://answers.opencv.org/question/186217/composert-inputoutput-question/ I have two frame transformations, one from frame 0 to frame 1, and one from frame 1 to frame 2, and I would like to concatenate these into the transformation from frame 0 to frame 2. I'm computing the pose of a moving camera, and the first transformation (0-1) represent the previous pose of the camera (transformation from its initial pose) and the 1-2 transformation is the newest change of pose, represented by tvec and rvec gotten from solvePnPRansac.
However, as I cannot just try out different inputs in my code and see if the output seems correct, since my system currently consists of lots of noise, I would like to have the math check out before I implement it into my application. But while I try to use the formulas given in the documentation with different rvecs and tvecs, I can't get the outputs (rvec3/tvec3) I want. These are the formulas:
![image description](/upfiles/15204294873689042.png)
I've tried with the following rvecs/tvecs:
- rvec1: Rotation *from* frame 1 *to* frame 0
- tvec1: vector *from* the origin of frame 0 *to* the origin of frame 1, given in *frame 0 coordinates*
- rvec2: Rotation *from* frame 2 *to* frame 1
- tvec2: vector *from* the origin of frame 1 *to* the origin of frame 2, given in *frame 1 coordinates*
I want to end up with:
- rvec3: Rotation from frame 0 to frame 2 (or inversed, doesn't matter)
- tvec3: vector *from* frame 0 *to* frame 2 given in *frame 0 coordinates* (or negated)
However, with these vectors, I can't get the formula in the documentation to make sense. The rvec3-formula makes sense, and with rvec1/rvec2 I get rvec3=(rvec2*rvec1)⁻¹ to equal the rotation from frame 2 to frame 0. However, the computation of tvec3 doesn't add up:
The formula says tvec3 = rvec2 * tvec1 + tvec2, but rvec2 * tvec1 doesn't make sense with my vectors. I mean, it says to rotate a vector given in frame 0 coordinates from frame 2 to frame 1. A vector given in frame 0 coordinates need to be multiplied with a rotation matrix representing the rotation *from* frame 0 to some other frame, but this is not the case here. And it haven't made sense with any other vectors I've tried as well, for that matter.
Someone that could help me with these calculcations? Thanks!
bendikivWed, 07 Mar 2018 07:38:10 -0600http://answers.opencv.org/question/186217/Stereo camera pose estimation from solvePnPRansac using 3D points given wrt. the camera coordinate systemhttp://answers.opencv.org/question/185671/stereo-camera-pose-estimation-from-solvepnpransac-using-3d-points-given-wrt-the-camera-coordinate-system/I know that there exists many posts regarding pose estimation using solvePnP/solvePnPRansac, and I've read most of them, but my case differs slightly from what seems to be the standard scenario, and even if I think I've got it I just can't seem to get it to work and would like someone to correct me if I'm doing anything wrong. This post became quite long, but please bear with me.
I'm trying to use solvePnPRansac to calculate the motion of a stereo camera from one frame/time instance to the next. I detect features in the first frame and track them to the next frame. I'm also using a stereo camera that comes with a SDK which provides me with the corresponding 3D coordinates of the detected features. **These 3D points are wrt. the camera coordinate system**. In other words, the 2D/3D-points in the two consecutive frames are corresponding to the same features, but if the camera moves between the frames, the coordinates change (even the 3D points, since they are relative to the camera origin).
I believe that the 3D input points of solvePnPRansac should be wrt a world frame, but since I don't have a world frame, I try to do the following:
1) For the very first frame: I set the initial camera pose as the world frame, since I need a constant reference for computation of relative movement. This means that the 3D points calculated in this frame now equals the world points, and that the movement of the camera is relative to the initial camera pose.
2) Call solvePnPRansac with the world points from the first frame together with the 2D features detected in the second frame as inputs. It returns rvec and tvec
**Now for my first question:** Is tvec the vector from the camera origin (/the second frame) to the world origin (/the first frame), given in the camera's coordinates system?
**Second question:** I want the vector from the world frame to the camera/second frame given in world frame coordinates (this should be equal to the translation of the camera relative to the original pose=world frame), so I need to use **translation = -(R)^T * tvec**, where R is the rotation matrix given by rvec?
Now I'm a little confused as to which 3D points I should use in the further calculations. Should I transform the 3D points detected in the second frame (which is given wrt the camera) to the world frame? If I combine the tvec and rvec into a homogeneous-transformation matrix T (which would represent the homogeneous transformation from the world frame to the second frame), the transformation should be
**3Dpoints_at_frame2_in_worldcoordinates = T^(-1) * 3Dpoints_at_frame2_in_cameracoordinates**
If I do this, I can capture a new image (third frame), track the 2D features detected in the second frame to the third frame, compute the corresponding 3D points (which is given wrt the third frame) and call solvePnPRansac with "3Dpoints_at_frame2_in_worldcoordinates" and the 2D features at the third frame as input. The returned rvec and tvec represents a conversion of world points to the third frame, i. e. if I use the same formula as in my second question I would get the absolute movement from the world frame to the third frame. And if I create a new homogeneous-transformation matrix I can convert the 3D points at the third frame into world coordinates and use these points in the next iteration (fourth frame). **Does this make any sense?**
Any answers or attempts on pointing out things I've might misunderstood (or understood, for that matter) are greatly appreciated!bendikivWed, 28 Feb 2018 04:49:20 -0600http://answers.opencv.org/question/185671/'Mapping' Aruco markers?http://answers.opencv.org/question/175705/mapping-aruco-markers/I have a calibrated camera, and an application that tracks Aruco markers using opencv 3.2.
What I am trying to do is Map the locations of the markers as they enter camera frame, in real time. So, for example, i have three markers.
In frame 1, I can see marker A. I want this to be located at 0,0,0 in world space.
I move the camera and can now see marker A and marker B.
I move the camera again, and can now see marker B and marker C, but no longer marker A.
What I want to do, is get the location and rotation of each marker in world space, relative to each other. It seems like I should have enough information to do this, but I am struggling to work out the workflow.
What I am doing is:
Keeping a vector of Markers, and keeping track of which I have seen before and which i have not.
As a new marker comes in to view, I store the rvec and tvec and flip it from camera pose to get the marker location in world space.
This alone does not do what I need, it seems to give the marker location relative to the camera, but not relative to each other and therefore the world. I assume that i need to somehow 'chain' the marker locations to get the world-space location of the new ones as the camera moves. is this correct? If anyone could give me a workflow, or steps to follow here, it would be very much appreciated.
thank you.stillNoviceWed, 04 Oct 2017 13:43:11 -0500http://answers.opencv.org/question/175705/Camera pose from homography?http://answers.opencv.org/question/166242/camera-pose-from-homography/Given K, an intrinsic camera matrix, a reference image from a camera 1 which pose is known and an image from a camera 2 which pose is unknown, is there a way to compute the pose of camera 2 using the homography matrix found between the two images from matched key-points if I know their 3D coordinates (these points may not be coplanar at all)? Or, if not, from anyone of the fundamental or essential matrices?
Could it perform better or faster thant SolvePnP?
swiss_knightSat, 15 Jul 2017 02:05:54 -0500http://answers.opencv.org/question/166242/aruco giving strange rotations.http://answers.opencv.org/question/143072/aruco-giving-strange-rotations/Hi, I am tracking aruco markers in an openCv application.
I am using the left frame of a calibrated stereo camera, so the image is undistorted and rectified. As such, i pass 0 parameters to the distortion matrix.
I have the camera pointing down at the floor, and measure the angle with an IMU. The result is 37 degrees in pitch.
I have a marker set up, on the floor, as straight as I can get it in front of the camera.
When I print the rvecs value returned from the marker, I would expect to see something like:
Yaw 0, Roll 0, Pitch -37
but what I do see ( after applying *180 / M_PI;) is:
[131.865, -0.295742, 1.04376]
The rotation is only one axis, which looks right, but the value is a long way off from what it should be. Am I missing something? Or is this as accurate as Aruco gets with a single marker?
(the axis displayed in frame looks correct, but the value returned does not)
As a bonus question.. is it possible to return quaternions from Aruco tracking?
Thanks!
EDIT:
I now have:
cv::Mat expected;
cv::Rodrigues(rvecs[i], expected);
Eigen::Quaterniond mOrientationEigen = toQuaternionOg(expected);
Eigen::Quaterniond toQuaternionOg(const cv::Mat &M)
{
Eigen::Matrix<double, 3, 3> eigMat = toMatrix3d(M);
Eigen::Quaterniond q(eigMat);
return q;
}
But the returned quaternions are garbage:
`3.89581e-181.20234e+173.03257e-186.61337e+09`
Where am I going wrong here?
antithingSat, 22 Apr 2017 06:38:04 -0500http://answers.opencv.org/question/143072/Will the distribution of 3D coordinates affect the accuracy of solvePnP?http://answers.opencv.org/question/134351/will-the-distribution-of-3d-coordinates-affect-the-accuracy-of-solvepnp/I am still trying to figure out a proper way to use solvePnP(). Can someone tell me what if input 3D coordinates for solvePnP() is not regular distributed, will the returned pose estimation still accurate? The experiment I did is assigning 3D coordinates to randomly distributed feature points on a poster and take a sequence of pictures while I move straight toward the poster. The resulted camera's XYZ coordinates are a bit jumpy.
For the record, I did conduct the following operations to derive camera pose.
Mat R;
Rodrigues(rvec, R);
R = R.t();
tvec = -R*tvec;
By the way, the 3D coordinates values that I assigned are a bit high. Some values are range over couple thousand while some are couple hundreds.
yorkhuangThu, 16 Mar 2017 08:42:36 -0500http://answers.opencv.org/question/134351/Does anyone actually use the return values of solvePnP() to compute the coordinates of the camera w.r.t. the object in the world space?http://answers.opencv.org/question/133855/does-anyone-actually-use-the-return-values-of-solvepnp-to-compute-the-coordinates-of-the-camera-wrt-the-object-in-the-world-space/ All the demo video of pose estimation using solvePnP() given in various posts exhibit a wireframe coordinate system or a wireframe object on top of the target image only. Does anyone actually use the return values of solvePnP() to compute the coordinates of the camera w.r.t. the object in the world space? My main confuse is the return values from solvePnP() are the rotation and translation of the **object** in **camera coordinate system**. Can we actually use return values to compute camera pose w.r.t. the object in the world space?
I been searching this answer for over two months. Can anyone help me? Thanks,yorkhuangMon, 13 Mar 2017 08:04:22 -0500http://answers.opencv.org/question/133855/