OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Fri, 12 Jul 2019 05:23:48 -0500ArUco orientation using the function aruco.estimatePoseSingleMarkers()http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/Hi everyone!
I'm trying to program a python app that determine the position and orientation of an aruco marker. I calibrated the camera and everything and I used *aruco.estimatePoseSingleMarkers* that returns the translation and rotation vectors.
The translation vector works fine but I don't understand how the rotation vector works. I took some picture to illustrate my problem with the "roll rotation":
Here the rotation vector is approximately [in degree]: [180 0 0]
![image description](/upfiles/15626850347225475.png)
Here the rotation vector is approximately [in degree]: [123 -126 0]
![image description](/upfiles/15626851829885092.png)
And here the rotation vector is approximately [in degree]: [0 -180 0]
![image description](/upfiles/15626853815019584.png)
And I don't see the logic in these angles. I've tried the other two rotations (pitch and yaw) and there appear also "random". So if you have an explication I would be very happy :) Tue, 09 Jul 2019 10:28:00 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/Comment by lamaa for <p>Hi everyone!</p>
<p>I'm trying to program a python app that determine the position and orientation of an aruco marker. I calibrated the camera and everything and I used <em>aruco.estimatePoseSingleMarkers</em> that returns the translation and rotation vectors.
The translation vector works fine but I don't understand how the rotation vector works. I took some picture to illustrate my problem with the "roll rotation":</p>
<p>Here the rotation vector is approximately [in degree]: [180 0 0]
<img alt="image description" src="/upfiles/15626850347225475.png"></p>
<p>Here the rotation vector is approximately [in degree]: [123 -126 0]
<img alt="image description" src="/upfiles/15626851829885092.png"></p>
<p>And here the rotation vector is approximately [in degree]: [0 -180 0]
<img alt="image description" src="/upfiles/15626853815019584.png"></p>
<p>And I don't see the logic in these angles. I've tried the other two rotations (pitch and yaw) and there appear also "random". So if you have an explication I would be very happy :) </p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215381#post-id-215381Yes but I rounded and put them in degree to have a better idea of there meaningTue, 09 Jul 2019 10:40:56 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215381#post-id-215381Comment by berak for <p>Hi everyone!</p>
<p>I'm trying to program a python app that determine the position and orientation of an aruco marker. I calibrated the camera and everything and I used <em>aruco.estimatePoseSingleMarkers</em> that returns the translation and rotation vectors.
The translation vector works fine but I don't understand how the rotation vector works. I took some picture to illustrate my problem with the "roll rotation":</p>
<p>Here the rotation vector is approximately [in degree]: [180 0 0]
<img alt="image description" src="/upfiles/15626850347225475.png"></p>
<p>Here the rotation vector is approximately [in degree]: [123 -126 0]
<img alt="image description" src="/upfiles/15626851829885092.png"></p>
<p>And here the rotation vector is approximately [in degree]: [0 -180 0]
<img alt="image description" src="/upfiles/15626853815019584.png"></p>
<p>And I don't see the logic in these angles. I've tried the other two rotations (pitch and yaw) and there appear also "random". So if you have an explication I would be very happy :) </p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215380#post-id-215380what does "is approximately [in degree]" mean ? you get an (exact) vector of 3 angles in rad.Tue, 09 Jul 2019 10:37:17 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215380#post-id-215380Answer by Eduardo for <p>Hi everyone!</p>
<p>I'm trying to program a python app that determine the position and orientation of an aruco marker. I calibrated the camera and everything and I used <em>aruco.estimatePoseSingleMarkers</em> that returns the translation and rotation vectors.
The translation vector works fine but I don't understand how the rotation vector works. I took some picture to illustrate my problem with the "roll rotation":</p>
<p>Here the rotation vector is approximately [in degree]: [180 0 0]
<img alt="image description" src="/upfiles/15626850347225475.png"></p>
<p>Here the rotation vector is approximately [in degree]: [123 -126 0]
<img alt="image description" src="/upfiles/15626851829885092.png"></p>
<p>And here the rotation vector is approximately [in degree]: [0 -180 0]
<img alt="image description" src="/upfiles/15626853815019584.png"></p>
<p>And I don't see the logic in these angles. I've tried the other two rotations (pitch and yaw) and there appear also "random". So if you have an explication I would be very happy :) </p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?answer=215415#post-id-215415My first advice is to look at a course or a book on rigid body transformation, homogeneous transformation topics.
These topics are well-covered in a robotics or computer graphics course, for instance:
- [Ridig Body Motion – Homogeneous Transformations, Claudio Melchiorri](http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf)
---
You have to understand what `rvec` and `tvec` are. First the camera model used in computer vision is described [here](https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#details). Additional information on pose estimation are described [here](https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d).
---
`rvec` is a Rodrigues rotation vector, or [axis-angle representation](https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation).
![image description](/upfiles/15627474005529395.png)
Rotation direction is described by a unit vector and the "amount" of rotation by the length of the vector. This representation avoids issues you can have with Euler angles, i.e. [Gimbal lock](https://en.wikipedia.org/wiki/Gimbal_lock). Quaternion is another representation of rotation that don't have Gimbal lock issue.
**Most important, they are not Euler angles.**
---
Now, what `rvec` and `tvec` represent?
They allow transforming a 3D point expressed in one coordinate system to another one. Here, you transform 3D points expressed in the tag frame into the camera frame:
![image description](/upfiles/1562748098917871.png)
So, for the first case you should have (here I use the tag X-axis as an example):
![image description](/upfiles/15627482777940383.png)
For trivial case, you can recover the rotation matrix by looking at the values that allow transforming the X-axis, Y-axis and Z-axis of the tag into the camera frame.
---
In my opinion, it is very rare to need to use the Euler angles and I don't like them. Why?
First, because they are **12 different** [representations/conventions of Euler angles](https://en.wikipedia.org/wiki/Euler_angles#Rotation_matrix). So first you have to agree on which Euler convention you are using when you have to deal with other people / code.
Then, [Gimbal lock](https://en.wikipedia.org/wiki/Gimbal_lock).
You only need Euler angles when you have to be able to interpret the rotation. So only for printing or as an user input (e.g. to rotate a CAD model in Blender).
Rest of the time, computation should be / are done using the rotation matrix or quaternion representations.Wed, 10 Jul 2019 03:49:23 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?answer=215415#post-id-215415Comment by Eduardo for <p>My first advice is to look at a course or a book on rigid body transformation, homogeneous transformation topics.
These topics are well-covered in a robotics or computer graphics course, for instance:</p>
<ul>
<li><a href="http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf">Ridig Body Motion – Homogeneous Transformations, Claudio Melchiorri</a></li>
</ul>
<hr>
<p>You have to understand what <code>rvec</code> and <code>tvec</code> are. First the camera model used in computer vision is described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#details">here</a>. Additional information on pose estimation are described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d">here</a>.</p>
<hr>
<p><code>rvec</code> is a Rodrigues rotation vector, or <a href="https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation">axis-angle representation</a>.</p>
<p><img alt="image description" src="/upfiles/15627474005529395.png"></p>
<p>Rotation direction is described by a unit vector and the "amount" of rotation by the length of the vector. This representation avoids issues you can have with Euler angles, i.e. <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>. Quaternion is another representation of rotation that don't have Gimbal lock issue.</p>
<p><strong>Most important, they are not Euler angles.</strong></p>
<hr>
<p>Now, what <code>rvec</code> and <code>tvec</code> represent?</p>
<p>They allow transforming a 3D point expressed in one coordinate system to another one. Here, you transform 3D points expressed in the tag frame into the camera frame:</p>
<p><img alt="image description" src="/upfiles/1562748098917871.png"></p>
<p>So, for the first case you should have (here I use the tag X-axis as an example):</p>
<p><img alt="image description" src="/upfiles/15627482777940383.png"></p>
<p>For trivial case, you can recover the rotation matrix by looking at the values that allow transforming the X-axis, Y-axis and Z-axis of the tag into the camera frame.</p>
<hr>
<p>In my opinion, it is very rare to need to use the Euler angles and I don't like them. Why? </p>
<p>First, because they are <strong>12 different</strong> <a href="https://en.wikipedia.org/wiki/Euler_angles#Rotation_matrix">representations/conventions of Euler angles</a>. So first you have to agree on which Euler convention you are using when you have to deal with other people / code.</p>
<p>Then, <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>.</p>
<p>You only need Euler angles when you have to be able to interpret the rotation. So only for printing or as an user input (e.g. to rotate a CAD model in Blender).</p>
<p>Rest of the time, computation should be / are done using the rotation matrix or quaternion representations.</p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215562#post-id-215562The axes drawn is the way to be sure that the pose is correctly estimated (red is X-axis, green is Y-axis and blue the Z-axis). Check that the axes correspond to how you have defined the object 3D points.Fri, 12 Jul 2019 05:23:48 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215562#post-id-215562Comment by lamaa for <p>My first advice is to look at a course or a book on rigid body transformation, homogeneous transformation topics.
These topics are well-covered in a robotics or computer graphics course, for instance:</p>
<ul>
<li><a href="http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf">Ridig Body Motion – Homogeneous Transformations, Claudio Melchiorri</a></li>
</ul>
<hr>
<p>You have to understand what <code>rvec</code> and <code>tvec</code> are. First the camera model used in computer vision is described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#details">here</a>. Additional information on pose estimation are described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d">here</a>.</p>
<hr>
<p><code>rvec</code> is a Rodrigues rotation vector, or <a href="https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation">axis-angle representation</a>.</p>
<p><img alt="image description" src="/upfiles/15627474005529395.png"></p>
<p>Rotation direction is described by a unit vector and the "amount" of rotation by the length of the vector. This representation avoids issues you can have with Euler angles, i.e. <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>. Quaternion is another representation of rotation that don't have Gimbal lock issue.</p>
<p><strong>Most important, they are not Euler angles.</strong></p>
<hr>
<p>Now, what <code>rvec</code> and <code>tvec</code> represent?</p>
<p>They allow transforming a 3D point expressed in one coordinate system to another one. Here, you transform 3D points expressed in the tag frame into the camera frame:</p>
<p><img alt="image description" src="/upfiles/1562748098917871.png"></p>
<p>So, for the first case you should have (here I use the tag X-axis as an example):</p>
<p><img alt="image description" src="/upfiles/15627482777940383.png"></p>
<p>For trivial case, you can recover the rotation matrix by looking at the values that allow transforming the X-axis, Y-axis and Z-axis of the tag into the camera frame.</p>
<hr>
<p>In my opinion, it is very rare to need to use the Euler angles and I don't like them. Why? </p>
<p>First, because they are <strong>12 different</strong> <a href="https://en.wikipedia.org/wiki/Euler_angles#Rotation_matrix">representations/conventions of Euler angles</a>. So first you have to agree on which Euler convention you are using when you have to deal with other people / code.</p>
<p>Then, <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>.</p>
<p>You only need Euler angles when you have to be able to interpret the rotation. So only for printing or as an user input (e.g. to rotate a CAD model in Blender).</p>
<p>Rest of the time, computation should be / are done using the rotation matrix or quaternion representations.</p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215486#post-id-215486Yes your right, my bad, I don't need this transformation. But I'm still struggling with the orientation of the aruco... I converted rvec into quaternions and visualized them but they don't represent the way the aruco is oriented.Thu, 11 Jul 2019 04:08:03 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215486#post-id-215486Comment by Eduardo for <p>My first advice is to look at a course or a book on rigid body transformation, homogeneous transformation topics.
These topics are well-covered in a robotics or computer graphics course, for instance:</p>
<ul>
<li><a href="http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf">Ridig Body Motion – Homogeneous Transformations, Claudio Melchiorri</a></li>
</ul>
<hr>
<p>You have to understand what <code>rvec</code> and <code>tvec</code> are. First the camera model used in computer vision is described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#details">here</a>. Additional information on pose estimation are described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d">here</a>.</p>
<hr>
<p><code>rvec</code> is a Rodrigues rotation vector, or <a href="https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation">axis-angle representation</a>.</p>
<p><img alt="image description" src="/upfiles/15627474005529395.png"></p>
<p>Rotation direction is described by a unit vector and the "amount" of rotation by the length of the vector. This representation avoids issues you can have with Euler angles, i.e. <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>. Quaternion is another representation of rotation that don't have Gimbal lock issue.</p>
<p><strong>Most important, they are not Euler angles.</strong></p>
<hr>
<p>Now, what <code>rvec</code> and <code>tvec</code> represent?</p>
<p>They allow transforming a 3D point expressed in one coordinate system to another one. Here, you transform 3D points expressed in the tag frame into the camera frame:</p>
<p><img alt="image description" src="/upfiles/1562748098917871.png"></p>
<p>So, for the first case you should have (here I use the tag X-axis as an example):</p>
<p><img alt="image description" src="/upfiles/15627482777940383.png"></p>
<p>For trivial case, you can recover the rotation matrix by looking at the values that allow transforming the X-axis, Y-axis and Z-axis of the tag into the camera frame.</p>
<hr>
<p>In my opinion, it is very rare to need to use the Euler angles and I don't like them. Why? </p>
<p>First, because they are <strong>12 different</strong> <a href="https://en.wikipedia.org/wiki/Euler_angles#Rotation_matrix">representations/conventions of Euler angles</a>. So first you have to agree on which Euler convention you are using when you have to deal with other people / code.</p>
<p>Then, <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>.</p>
<p>You only need Euler angles when you have to be able to interpret the rotation. So only for printing or as an user input (e.g. to rotate a CAD model in Blender).</p>
<p>Rest of the time, computation should be / are done using the rotation matrix or quaternion representations.</p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215419#post-id-215419For inverse transformation, have a look at [this, p72](http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf). You don't need to compute the matrix inverse.
Not sure to understand what you need. In `(x,y,z,1)_cam = [R | t] (x,y,z,1)_tag`, `t` is translation of the tag frame wrt the camera frame in the camera coordinates system. That means that if you have `tx = 10 cm`, the tag center is at 10 cm in camera X-axis.
If you inverse the transformation, it is now the camera frame wrt the tag frame in the tag coordinates system.Wed, 10 Jul 2019 05:17:26 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215419#post-id-215419Comment by lamaa for <p>My first advice is to look at a course or a book on rigid body transformation, homogeneous transformation topics.
These topics are well-covered in a robotics or computer graphics course, for instance:</p>
<ul>
<li><a href="http://www-lar.deis.unibo.it/people/cmelchiorri/Files_Robotica/FIR_03_Rbody.pdf">Ridig Body Motion – Homogeneous Transformations, Claudio Melchiorri</a></li>
</ul>
<hr>
<p>You have to understand what <code>rvec</code> and <code>tvec</code> are. First the camera model used in computer vision is described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#details">here</a>. Additional information on pose estimation are described <a href="https://docs.opencv.org/4.1.0/d9/d0c/group__calib3d.html#ga549c2075fac14829ff4a58bc931c033d">here</a>.</p>
<hr>
<p><code>rvec</code> is a Rodrigues rotation vector, or <a href="https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation">axis-angle representation</a>.</p>
<p><img alt="image description" src="/upfiles/15627474005529395.png"></p>
<p>Rotation direction is described by a unit vector and the "amount" of rotation by the length of the vector. This representation avoids issues you can have with Euler angles, i.e. <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>. Quaternion is another representation of rotation that don't have Gimbal lock issue.</p>
<p><strong>Most important, they are not Euler angles.</strong></p>
<hr>
<p>Now, what <code>rvec</code> and <code>tvec</code> represent?</p>
<p>They allow transforming a 3D point expressed in one coordinate system to another one. Here, you transform 3D points expressed in the tag frame into the camera frame:</p>
<p><img alt="image description" src="/upfiles/1562748098917871.png"></p>
<p>So, for the first case you should have (here I use the tag X-axis as an example):</p>
<p><img alt="image description" src="/upfiles/15627482777940383.png"></p>
<p>For trivial case, you can recover the rotation matrix by looking at the values that allow transforming the X-axis, Y-axis and Z-axis of the tag into the camera frame.</p>
<hr>
<p>In my opinion, it is very rare to need to use the Euler angles and I don't like them. Why? </p>
<p>First, because they are <strong>12 different</strong> <a href="https://en.wikipedia.org/wiki/Euler_angles#Rotation_matrix">representations/conventions of Euler angles</a>. So first you have to agree on which Euler convention you are using when you have to deal with other people / code.</p>
<p>Then, <a href="https://en.wikipedia.org/wiki/Gimbal_lock">Gimbal lock</a>.</p>
<p>You only need Euler angles when you have to be able to interpret the rotation. So only for printing or as an user input (e.g. to rotate a CAD model in Blender).</p>
<p>Rest of the time, computation should be / are done using the rotation matrix or quaternion representations.</p>
http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215418#post-id-215418Thanks a lot for you answer, now it's a lot clearer! :) I was struggling to find information about computer vision. Yeah you're right they are really tricky those Euler angles and there is no consensus about their representation. I remember that our teacher almost forbade us to use them. So I will follow your advice and work with the rotation matrix using *cv2.Rodrigues(rvec)*.
So if I understand well, to found the coordinate of the tag, I can simply do the following : (x,y,z,1)_tag = inv(transform_matrix) (x,y,z,1)_camera?
Because basically I'm working on a "vision system" for a robot (ABB MR6400). The camera would be on the robot'arm and knowing it's tool coordinate, it would know the position of the tag.Wed, 10 Jul 2019 04:41:15 -0500http://answers.opencv.org/question/215377/aruco-orientation-using-the-function-arucoestimateposesinglemarkers/?comment=215418#post-id-215418