Ask Your Question
2

ArUco marker z-axis mirrored

asked 2017-05-04 10:03:02 -0500

eskrs gravatar image

updated 2017-05-08 06:39:01 -0500

We are using ArUco markers for pose estimation and are encountering an issue.

Fairly frequently a marker's z-axis will flip. First of all it is not even clear to me how this would be possible since a proper orthogonal rotation would not be able to mirror an axis by itself.

Why is this a valid output of ArUco? It would mean the marker is turned away from the camera which would make it impossible to detect in the first place. Is it possible to fix with ArUco parameters?

Turning the corner refinement on, introduces another issue: http://answers.opencv.org/question/14...

Even if you don't know how to fix the underlying issue, it would be helpful to suggest how to undo the glitch. Described in words, a fix would check whether the normal vector of the marker is pointing away from the camera and then reflect the transformation on the xy-plane of the marker. Intuitively this sounds easy, but the rotation representation is very difficult for me to reason about.

Below is a test program that takes a file name as argument and two example images to demonstrate the issue. The two images are two consecutive frames of a video and have very little differences.

#include "opencv2/opencv.hpp"
#include "opencv2/aruco.hpp"


int main(int argc, char *argv[])
{
    if (argc != 2)
        return 1;


    cv::Mat frame = cv::imread(argv[1]);
    cv::Mat debugFrame = frame.clone();

    // Intrinsic camera parameters.
    cv::Mat camMatrix = cv::Mat::eye(3, 3, CV_64F);
    cv::Mat distortionCoeffs = cv::Mat::zeros(8, 1, CV_64F);
    camMatrix.at<double>(0, 0) = 2.3396381685789738e+03;
    camMatrix.at<double>(0, 2) = 960.;
    camMatrix.at<double>(1, 1) = 2.3396381685789738e+03;
    camMatrix.at<double>(1, 2) = 540.;
    distortionCoeffs.at<double>(0, 0) = -1.0982746232841779e-01;
    distortionCoeffs.at<double>(1, 0) = 2.2689585715220828e-01;
    distortionCoeffs.at<double>(2, 0) = 0.;
    distortionCoeffs.at<double>(3, 0) = 0.;
    distortionCoeffs.at<double>(4, 0) = -2.2112148171171589e-01;

    cv::Ptr<cv::aruco::Dictionary> dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_4X4_50);
    cv::Ptr<cv::aruco::DetectorParameters> params = cv::aruco::DetectorParameters::create();
    std::vector<int> markerIds;
    std::vector<std::vector<cv::Point2f>> markerCorners;
    std::vector<cv::Vec3d> rotationVec, translationVec;


    params->doCornerRefinement = false;
    cv::aruco::detectMarkers(frame, dict, markerCorners, markerIds, params);
    auto numMarkers = markerIds.size();

    if (numMarkers > 0)
    {
        cv::aruco::drawDetectedMarkers(debugFrame, markerCorners, markerIds);

        cv::aruco::estimatePoseSingleMarkers(markerCorners, 5.0, camMatrix, distortionCoeffs, rotationVec, translationVec);

        for (int i = 0; i < numMarkers; i++)
        {
            // Official aruco axis drawing.
            cv::aruco::drawAxis(debugFrame, camMatrix, distortionCoeffs, rotationVec[i], translationVec[i], 0.5 * 5.0);


            // Project marker origin.
            std::vector< cv::Point3f > axisPoints;
            axisPoints.push_back(cv::Point3f(0, 0, 0));
            std::vector< cv::Point2f > imagePoints;
            cv::projectPoints(axisPoints, rotationVec[i], translationVec[i], camMatrix, distortionCoeffs, imagePoints);

            // Convert rotation vector to rotation matrix.
            cv::Mat rotMat;
            cv::Rodrigues(rotationVec[i], rotMat);

            // Check for proper rotation matrix.
            assert(cv::determinant(rotMat) > 0.99 && cv::determinant(rotMat) < 1.01);

            // Manually rotate the z-axis vector.
            cv::Mat testVec(cv::Point3f(0, 0, 10.0f));
            testVec.convertTo(testVec, CV_64FC1);
            cv::Mat testTransformed ...
(more)
edit retag flag offensive close merge delete

Comments

@eskrs I believe it is just an optical illusion. For the last image, the marker axis is right-handed but the z-axis is not going downward or inside the table but toward us. The x-axis is going more or less inside the table. It is easier to see that by looking at the cube where the dimension does not fit if the rotation matrix was left-handed.

See also.

Pose estimation is not a trivial problem. Noise in the measurement or in the intrinsic parameters could lead to issue like that in my opinion.

Eduardo gravatar imageEduardo ( 2018-01-19 14:34:36 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-05-04 15:58:36 -0500

LBerger gravatar image

updated 2017-05-04 16:08:40 -0500

First I use opencv-3.2.0-dev and I don't think you use same version. I cannot compiled this line using VS 2015 :

params->doCornerRefinement = false;

I changed it in

params->cornerRefinementMethod = 0;

with this line I found same results with those values for R and T :

 Corners =[988, 512;
 945, 575;
 849, 544;
 893, 480]
 R = [1.79273, 2.85307, 1.24564]
 T=[-1.96633, -0.555819, 111.496]
 Corners =[989, 510;
 947, 573;
 850, 543;
 894, 479]
 R = [1.36133, 2.10818, -0.902147]
 T=[-1.90003, -0.681505, 111.217]

But now If I write :

params->cornerRefinementMethod = cv::aruco::CORNER_REFINE_CONTOUR;

with this line I found same results with those values for R and T :

 Corners =[987.86407, 511.47787;
 945.42108, 574.61035;
 848.55096, 543.72046;
 893.2522, 480.44275]
 R = [1.35387, 2.11157, -0.904189]
 T=[-1.95807, -0.619014, 111.203]
 Corners =[988.87256, 510.07428;
 946.534, 573.24799;
 849.7066, 542.39569;
 894.38611, 479.09317]
 R = [1.3551, 2.11294, -0.903394]
 T=[-1.90753, -0.684269, 111.31]

Axis is http://answers.opencv.org/upfiles/149... for both images. Update your repo and try to use params->cornerRefinementMethod = cv::aruco::CORNER_REFINE_CONTOUR;

edit flag offensive delete link more

Comments

Thanks for the answer. This does improve the overall performance, but introduces another issue: http://answers.opencv.org/question/14...

eskrs gravatar imageeskrs ( 2017-05-08 06:37:47 -0500 )edit

Now I was even able to reproduce the same issue with corner refinement, so this does not solve the underlying issue of impossible poses. http://i.imgur.com/XpNTHYp.pnghttp://i.imgur.com/n6OIU0L.png

eskrs gravatar imageeskrs ( 2017-05-08 06:57:14 -0500 )edit
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2017-05-04 10:03:02 -0500

Seen: 1,338 times

Last updated: May 08 '17