I calibrated my mono camera using opencv. Now I know the camera intrinsic matrix and distortion coefs [K1, K2, P1 ,P2,K3 ,K4, K5, K6] of my camera. Assuming that camera is place in [x, y, z] with [Roll, Pitch, Yaw] rotations. how can I get each pixel in world coordinate when the camera is looking on the floor [z=0].
update: I made it an answer since it was not fitting in a comment, and multiple comments with code were ugly and not helping to understand
@Theoretiker you need to obtain the tvec, rvec and rotationMatrix. the first two you get them through the solvePnP() function and the latter through the Rodrigues() function. Once you have this information in addition to the camera calibration coefficients you can transform the pixel point to world coordinates point. The code should be something like that (it is taken from an old project of mine):
Mat intrinsics, distCoeffs;
rvec.create(1,3,cv::DataType<double>::type);
tvec.create(1,3,cv::DataType<double>::type);
rotationMatrix.create(3,3,cv::DataType<double>::type);
cv::solvePnP(worldPlane, imagePlane, intrinsics, distCoeffs, rvec, tvec);
cv::Rodrigues(rvec,rotationMatrix);
cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type); //u,v,1
// image point
uvPoint.at<double>(0,0) = 3.; //got this point using mouse callback
uvPoint.at<double>(1,0) = 134.;
cv::Mat tempMat, tempMat2;
double s, zConst = 0;
tempMat = rotationMatrix.inv() * intrinsics.inv() * uvPoint;
tempMat2 = rotationMatrix.inv() * tvec;
s = zConst + tempMat2.at<double>(2,0);
s /= tempMat.at<double>(2,0);
cv::Mat wcPoint = rotationMatrix.inv() * (s * intrinsics.inv() * uvPoint - tvec);
Point3f realPoint(wcPoint.at<double>(0, 0), wcPoint.at<double>(1, 0), wcPoint.at<double>(2, 0)); // point in world coordinates
So you basically invert the pinhole camera equation and find out s such that the ray intersects with the z = 0 plane?
Thanks for your complete answer. Is it possible that instead of using solvepnp I fill rvec and tvec? Based on IMU data?
I have the rotation [Roll, Pitch, Yaw] and transition from the Z=0 plane which is [X, Y, Z].
If it is possible could you briefly tell me how can i fill rvec and tvec?
@Theoretiker, something like that. @mefmef I do not know/remember that, the project that I worked with it was 2 years ago, and since then there was not need to work with it again so some of the information/knowledge faded out. But try to search with the keywords that I gave you at the comment in your opening question. I still remember that there was a lot of material, with examples as well, to read about it when I was searching at that point.
Hello.
I did this, and also this related article on StackOverflow: https://stackoverflow.com/questions/1...
with that same image (its the image described there), and I can't get the positions (realPoints) right!
And of course, when I proyected them back, they are well off.
Also, in :
s = zConst + tempMat2.at<double>(2,0);
s /= tempMat.at<double>(2,0);
The "2" is because is z-constant? its the z-pos in the matrix? being z the vertical axis, zeroed at floor?
Asked: 2015-05-28 04:32:38 -0600
Seen: 16,573 times
Last updated: May 28 '15
I'd be interested in this as well. I need to know which ray each point on the image corresponds to. Basically I would apply the answer to your question twice for the front and back plane of the box I look at the connect them. Your question should be answerable with a bit of linear algebra if one neglects the curvature. I'll play around in Mathematica in the meanwhile.
hints: search for pose estimation, solvePnP(), Rodrigues() in addition to image coordinates to world coordinates and opencv...and be prepared for a lot of reading.
@theodore: As far as I have understood one needs
solvePnP()to get the position and angle of the camera in world coordinates. Once one has this, how does this function help? I am currently thinking about taking a random point on the plane and project it to the image and then iteratively see which points is closest to the given image point.