ArUco tracking in OpenCV
I have been working on ArUco detection and tracking to get the location of moving camera which is at a distance from the marker(static marker).
I used cv::solvePnP() to achieve the pose of the marker and ultimately got the pose of the of the camera (Z axis distance). I followed this How to calculate the angle from rotation matrix to get the yaw, roll and pitch of the camera. The roll angle results are stable. But I see the yaw angle results are unstable. I have the following questions:
- As the distance of the camera from the marker increases would there be any instability in detecting the corners? I am using a fairly large sized marker.
- The yaw angle results seem to be alright when camera is closer to the marker. But as the distance increases the "Z" axis flips giving me the inverted sign values beyond a certain point. How do I go about this?
- Are there any other possible ways to calculate the yaw angle of the camera which would lead me to stable results?
The code snippet:
cv::solvePnP(objectPoints,imagePoints,camMatrix,distCoeffs,rvecs1,tvecs1,CV_ITERATIVE);
cv::Mat rotate;
cv::Rodrigues(rvecs1,rotate);
cv::Mat cam = rotate.t(); // rotation matrix
cv::Mat camdist = -cam * tvecs1; // camera pose
double m11 = cam.at<double>(0, 0);
double m12 = cam.at<double>(0,1);
double m13 = cam.at<double>(0, 2);
double m21 = cam.at<double>(1, 0);
double m22 = cam.at<double>(1, 1);
double m23 = cam.at<double>(1, 2);
double m31 = cam.at<double>(2, 0);
double m32 = cam.at<double>(2,1);
double m33 = cam.at<double>(2, 2);
double yaw = atan2(-m31,cv::sqrt((m32*m32) + (m33*m33)));
Any help would be greatly appreciated.
Please show your code so others can know exactly what you have tried.
I updated the question with code
Have you tried using
cv::aruco::detectMarkers()andcv::aruco::estimatePoseBoard()(or something similar)?I tried
cv::aruco::estimatePoseSingleMarkers(). It gives me same exact results ascv::solvePnP().Try using
cv::RQDecomp3x3()orcv::decomposeProjectionMatrix()for obtaining euler angles. And remember that the results are initially in the coordinate system of the camera as (0,0,0,0,0,0). To get the camera pose using the marker as the origin of the coordinate system you'll want to invert the rotation and negate the translation.Thank you for your reply.I already have the euler angles. I see stable results for the position of camera. My issues is the yaw angles of camera when moved are highly unstable. Can I use Kalman Filter to reduce the noise? If yes how?
I would try the alternative methods mentioned above for getting euler angles, perhaps they offer numerical stability? Verify that they give nearly similar results to your method. Also, you might want to use a larger marker from the smallest dictionary, and track at the highest camera resolution. I would think that would give you best results and still run at super frame rate. My (somewhat limited) experience with ArUco markers has been that for "reasonable" pixels-on-target the pose results are so stable there is no need for filtering.
Would having multiple markers of the same length/size help in stabilizing values?
Yes, it likely would. Try using an ArUco board. There is a utility program provided with OpenCV to create a board image.
Hi I tried implementing aruco board and I'm facing an issue I can't resolve. I don't know where I'm going wrong and I am just a rookie in computer vision and c++. I stuck a bunch of markers(6) on a cardboard and specified the object points of all the corners 24 in total like this: std::vector<cv::point3f>objectPointsboard; objectPointsboard.push_back(cv::Point3f(0, 0, 0)); objectPointsboard.push_back(cv::Point3f(0.102, 0, 0));
objectPointsboard.push_back(cv::Point3f(0.102, 0.102, 0));
objectPointsboard.push_back(cv::Point3f(0, 0.102, 0)); ...
I am able to create a board object. But "estimateposeboard" fails. This is the error objectPoints.total()==ids.total() in function 'create'