ArUco tracking in OpenCV

asked 2018-08-03 02:18:52 -0500

slayer2309 gravatar image

updated 2018-08-03 16:03:19 -0500

I have been working on ArUco detection and tracking to get the location of moving camera which is at a distance from the marker(static marker). I used cv::solvePnP() to achieve the pose of the marker and ultimately got the pose of the of the camera (Z axis distance). I followed this How to calculate the angle from rotation matrix to get the yaw, roll and pitch of the camera. The roll angle results are stable. But I see the yaw angle results are unstable. I have the following questions:

  1. As the distance of the camera from the marker increases would there be any instability in detecting the corners? I am using a fairly large sized marker.
  2. The yaw angle results seem to be alright when camera is closer to the marker. But as the distance increases the "Z" axis flips giving me the inverted sign values beyond a certain point. How do I go about this?
  3. Are there any other possible ways to calculate the yaw angle of the camera which would lead me to stable results?

The code snippet:

cv::solvePnP(objectPoints,imagePoints,camMatrix,distCoeffs,rvecs1,tvecs1,CV_ITERATIVE);
cv::Mat rotate;
cv::Rodrigues(rvecs1,rotate);
cv::Mat cam = rotate.t(); // rotation matrix
cv::Mat camdist = -cam * tvecs1; // camera pose 
double m11 = cam.at<double>(0, 0);
double m12 = cam.at<double>(0,1);
double m13 = cam.at<double>(0, 2);
double m21 = cam.at<double>(1, 0);
double m22 = cam.at<double>(1, 1);
double m23 = cam.at<double>(1, 2);
double m31 = cam.at<double>(2, 0);
double m32 = cam.at<double>(2,1);
double m33 = cam.at<double>(2, 2);
double yaw = atan2(-m31,cv::sqrt((m32*m32) + (m33*m33)));

Any help would be greatly appreciated.

edit retag flag offensive close merge delete

Comments

Please show your code so others can know exactly what you have tried.

Der Luftmensch gravatar imageDer Luftmensch ( 2018-08-03 08:57:31 -0500 )edit

I updated the question with code

slayer2309 gravatar imageslayer2309 ( 2018-08-03 10:55:25 -0500 )edit

Have you tried using cv::aruco::detectMarkers() and cv::aruco::estimatePoseBoard() (or something similar)?

Der Luftmensch gravatar imageDer Luftmensch ( 2018-08-03 16:06:05 -0500 )edit

I tried cv::aruco::estimatePoseSingleMarkers(). It gives me same exact results as cv::solvePnP().

slayer2309 gravatar imageslayer2309 ( 2018-08-04 01:26:06 -0500 )edit

Try using cv::RQDecomp3x3() or cv::decomposeProjectionMatrix() for obtaining euler angles. And remember that the results are initially in the coordinate system of the camera as (0,0,0,0,0,0). To get the camera pose using the marker as the origin of the coordinate system you'll want to invert the rotation and negate the translation.

Der Luftmensch gravatar imageDer Luftmensch ( 2018-08-04 09:57:25 -0500 )edit

Thank you for your reply.I already have the euler angles. I see stable results for the position of camera. My issues is the yaw angles of camera when moved are highly unstable. Can I use Kalman Filter to reduce the noise? If yes how?

slayer2309 gravatar imageslayer2309 ( 2018-08-04 12:32:19 -0500 )edit

I would try the alternative methods mentioned above for getting euler angles, perhaps they offer numerical stability? Verify that they give nearly similar results to your method. Also, you might want to use a larger marker from the smallest dictionary, and track at the highest camera resolution. I would think that would give you best results and still run at super frame rate. My (somewhat limited) experience with ArUco markers has been that for "reasonable" pixels-on-target the pose results are so stable there is no need for filtering.

Der Luftmensch gravatar imageDer Luftmensch ( 2018-08-05 08:16:32 -0500 )edit

Would having multiple markers of the same length/size help in stabilizing values?

slayer2309 gravatar imageslayer2309 ( 2018-08-08 06:40:44 -0500 )edit

Yes, it likely would. Try using an ArUco board. There is a utility program provided with OpenCV to create a board image.

Der Luftmensch gravatar imageDer Luftmensch ( 2018-08-08 10:49:01 -0500 )edit

Hi I tried implementing aruco board and I'm facing an issue I can't resolve. I don't know where I'm going wrong and I am just a rookie in computer vision and c++. I stuck a bunch of markers(6) on a cardboard and specified the object points of all the corners 24 in total like this: std::vector<cv::point3f>objectPointsboard; objectPointsboard.push_back(cv::Point3f(0, 0, 0)); objectPointsboard.push_back(cv::Point3f(0.102, 0, 0));
objectPointsboard.push_back(cv::Point3f(0.102, 0.102, 0));
objectPointsboard.push_back(cv::Point3f(0, 0.102, 0)); ...

I am able to create a board object. But "estimateposeboard" fails. This is the error objectPoints.total()==ids.total() in function 'create'

slayer2309 gravatar imageslayer2309 ( 2018-08-14 08:03:32 -0500 )edit