OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Wed, 12 Feb 2020 03:25:26 -0600Is cv::triangulatePoints() returning 3D points in world coordinate system?http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/Considering a moving camera with fixed calibration matrix (intrinsic parameters), I am triangulating tracked feature points from two views that are not consecutive. The view poses are in camera coordinate system and images are already undistorted before detecting and tracking features.
Please can you confirm if the triangulated points are in world coordinate system after applying the cv::triangulatePoints() and cv::convertPointsFromHomogeneous() functions.Wed, 14 Dec 2016 08:54:28 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/Answer by Eduardo for <p>Considering a moving camera with fixed calibration matrix (intrinsic parameters), I am triangulating tracked feature points from two views that are not consecutive. The view poses are in camera coordinate system and images are already undistorted before detecting and tracking features.</p>
<p>Please can you confirm if the triangulated points are in world coordinate system after applying the cv::triangulatePoints() and cv::convertPointsFromHomogeneous() functions.</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?answer=119066#post-id-119066 The function [cv::triangulatePoints()](http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#gad3fc9a0c82b08df034234979960b778c) will compute, reconstruct the 3D points in the camera frame (the left camera frame should be the reference, to be checked).
Also, from my knowledge, this function should work only for a stereo camera setup where the two image views are fronto-parallel (the images are rectified with [cv::stereoRectify()](http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6)):
![image description](/upfiles/148175071642981.jpg)
One last thing, `cv::triangulatePoints()` needs the two projection matrices of the two cameras, that is the transformation between the left and right camera frames. In case of I am wrong and it is possible to use `cv::triangulatePoints()` with non fronto-parallel views, you will still need to have the relationship, the transformation matrix between the two camera frames (the intrinsic matrix and the pairs of points are not sufficient).Wed, 14 Dec 2016 15:33:29 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?answer=119066#post-id-119066Comment by mfischer-gundlach for <p>The function <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#gad3fc9a0c82b08df034234979960b778c">cv::triangulatePoints()</a> will compute, reconstruct the 3D points in the camera frame (the left camera frame should be the reference, to be checked).</p>
<p>Also, from my knowledge, this function should work only for a stereo camera setup where the two image views are fronto-parallel (the images are rectified with <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6">cv::stereoRectify()</a>):</p>
<p><img alt="image description" src="/upfiles/148175071642981.jpg"/></p>
<p>One last thing, <code>cv::triangulatePoints()</code> needs the two projection matrices of the two cameras, that is the transformation between the left and right camera frames. In case of I am wrong and it is possible to use <code>cv::triangulatePoints()</code> with non fronto-parallel views, you will still need to have the relationship, the transformation matrix between the two camera frames (the intrinsic matrix and the pairs of points are not sufficient).</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=226156#post-id-226156When your projection matrix is computed with an (R, t) that changes coordinates from world to camera coordinates, then the 3D points are expressed in the world coordinate system. Likewise, when you use the (R, t) from stereo calibration, that uses the first camera as origin, then the points from triangulation are expressed in the coordinate system of the first camera.
I ran some test, see the very last section of https://gist.github.com/FischerGundlach/b6b383e32e977ca0bc0de7121d316845Wed, 12 Feb 2020 03:25:26 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=226156#post-id-226156Comment by vik748 for <p>The function <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#gad3fc9a0c82b08df034234979960b778c">cv::triangulatePoints()</a> will compute, reconstruct the 3D points in the camera frame (the left camera frame should be the reference, to be checked).</p>
<p>Also, from my knowledge, this function should work only for a stereo camera setup where the two image views are fronto-parallel (the images are rectified with <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6">cv::stereoRectify()</a>):</p>
<p><img alt="image description" src="/upfiles/148175071642981.jpg"/></p>
<p>One last thing, <code>cv::triangulatePoints()</code> needs the two projection matrices of the two cameras, that is the transformation between the left and right camera frames. In case of I am wrong and it is possible to use <code>cv::triangulatePoints()</code> with non fronto-parallel views, you will still need to have the relationship, the transformation matrix between the two camera frames (the intrinsic matrix and the pairs of points are not sufficient).</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=208496#post-id-208496Can someone please confirm which coordinate frame the output 3D points are computed? Lets say P as shown in the diagram above is at the center of the baseline would its 3D coordinates be (0,0,Z) or (T/2,0,Z)?Wed, 06 Feb 2019 19:35:10 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=208496#post-id-208496Comment by carl777 for <p>The function <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#gad3fc9a0c82b08df034234979960b778c">cv::triangulatePoints()</a> will compute, reconstruct the 3D points in the camera frame (the left camera frame should be the reference, to be checked).</p>
<p>Also, from my knowledge, this function should work only for a stereo camera setup where the two image views are fronto-parallel (the images are rectified with <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6">cv::stereoRectify()</a>):</p>
<p><img alt="image description" src="/upfiles/148175071642981.jpg"/></p>
<p>One last thing, <code>cv::triangulatePoints()</code> needs the two projection matrices of the two cameras, that is the transformation between the left and right camera frames. In case of I am wrong and it is possible to use <code>cv::triangulatePoints()</code> with non fronto-parallel views, you will still need to have the relationship, the transformation matrix between the two camera frames (the intrinsic matrix and the pairs of points are not sufficient).</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=178646#post-id-178646Can you please guide me, how can I construct projection matrix (P0 and P1) from stereo calibration results? I have cammat1, cammat2, R and T. R and T are with respect to the first camera.Sat, 18 Nov 2017 03:02:44 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=178646#post-id-178646Comment by ale_Xompi for <p>The function <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#gad3fc9a0c82b08df034234979960b778c">cv::triangulatePoints()</a> will compute, reconstruct the 3D points in the camera frame (the left camera frame should be the reference, to be checked).</p>
<p>Also, from my knowledge, this function should work only for a stereo camera setup where the two image views are fronto-parallel (the images are rectified with <a href="http://docs.opencv.org/3.1.0/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6">cv::stereoRectify()</a>):</p>
<p><img alt="image description" src="/upfiles/148175071642981.jpg"/></p>
<p>One last thing, <code>cv::triangulatePoints()</code> needs the two projection matrices of the two cameras, that is the transformation between the left and right camera frames. In case of I am wrong and it is possible to use <code>cv::triangulatePoints()</code> with non fronto-parallel views, you will still need to have the relationship, the transformation matrix between the two camera frames (the intrinsic matrix and the pairs of points are not sufficient).</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=120911#post-id-120911thank you @Eduardo for your answer. The cv::triangulatePoints() does not require a stereo pair with fronto-parallel view (i.e. binocular), but you can apply the function with any pair of cameras given intrinsic and extrinsic parameters of each view and the feature points for each image. Of course, there are some degenerative configurations and the forward motion is the one that generates more uncertainty in the 3D points. However, after reading again HZ's Multi-View Geometry, the Hartley and Sturm's paper "Triangulation", and doing some experiments both in MATLAB and OpenCV, I can confirm that triangulated points are returned in world coordinate frame. Of course, if you fixed the first view as origin (of the world as well), then 3D points are given w.r.t. the first camera as you said.Mon, 02 Jan 2017 10:47:56 -0600http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?comment=120911#post-id-120911Answer by JankaSvK for <p>Considering a moving camera with fixed calibration matrix (intrinsic parameters), I am triangulating tracked feature points from two views that are not consecutive. The view poses are in camera coordinate system and images are already undistorted before detecting and tracking features.</p>
<p>Please can you confirm if the triangulated points are in world coordinate system after applying the cv::triangulatePoints() and cv::convertPointsFromHomogeneous() functions.</p>
http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?answer=187753#post-id-187753It depends on used projection matrices to call the function.
**Where to look for projection matrices?**
As <a href="https://docs.opencv.org/3.0-beta/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#triangulatepoints">OpenCV documentation</a> suggests you can use `stereoRectify()` which needs results from `calibrateCamera` and `stereoCalibration`. As far as I know, it chooses new coordinate system which has an origin between the cameras.
Another way to get projection matrices is to compute it by yourself. Let's assume that first camera position creates your world coordinate system. Projection matrix could be computed as **A * [R | T]**, where A is a instristic matrix for the camera and R is rotation vector and T translation vector. So for the first position you set as **rotation matrix identity** and as a **translation vector [0,0,0,1]<sup>T</sup>**. For the second projection matrix use the output of R, T from the `stereoCalibrate`.
I am working on locating an object in stereo vision, so far this approach gives me reasonable results. Please let me know if there is somewhere a problem.
Also I recommend to take a look at this <a href="https://stackoverflow.com/questions/16295551/how-to-correctly-use-cvtriangulatepoints/16299909?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa">question at StackOverflow</a>Mon, 26 Mar 2018 15:43:50 -0500http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/?answer=187753#post-id-187753