Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

update: I made it an answer since it was not fitting in a comment, and multiple comments with code were ugly and not helping to understand

@Theoretiker you need to obtain the tvec, rvec and rotationMatrix. the first two you get them through the solvePnP() function and the latter through the Rodrigues() function. Once you have this information in addition to the camera calibration coefficients you can transform the pixel point to world coordinates point. The code should be something like that (it is taken from an old project of mine):

Mat intrinsics, distCoeffs; rvec.create(1,3,cv::DataType<double>::type); tvec.create(1,3,cv::DataType<double>::type); rotationMatrix.create(3,3,cv::DataType<double>::type);

cv::solvePnP(worldPlane_v1, imagePlane, intrinsics, distCoeffs, rvec, tvec); cv::Rodrigues(rvec,rotationMatrix);

cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type); //u,v,1

// image point uvPoint.at<double>(0,0) = 3.; //got this point using mouse callback uvPoint.at<double>(1,0) = 134.;

cv::Mat tempMat, tempMat2; double s, zConst = 0; tempMat = rotationMatrix.inv() * intrinsics.inv() * uvPoint; tempMat2 = rotationMatrix.inv() * tvec; s = zConst + tempMat2.at<double>(2,0); //285 represents the height Zconst s /= tempMat.at<double>(2,0); cv::Mat wcPoint = rotationMatrix.inv() * (s * intrinsics.inv() * uvPoint - tvec);

Point3f realPoint(wcPoint.at<double>(0, 0), wcPoint.at<double>(1, 0), wcPoint.at<double>(2, 0)); // point in world coordinates

update: update: I made it an answer since it was not fitting in a comment, and multiple comments with code were ugly and not helping to understand

@Theoretiker you need to obtain the tvec, rvec and rotationMatrix. the first two you get them through the solvePnP() function and the latter through the Rodrigues() function. Once you have this information in addition to the camera calibration coefficients you can transform the pixel point to world coordinates point. The code should be something like that (it is taken from an old project of mine):

Mat intrinsics, distCoeffs;
rvec.create(1,3,cv::DataType<double>::type);
tvec.create(1,3,cv::DataType<double>::type);
rotationMatrix.create(3,3,cv::DataType<double>::type);

cv::solvePnP(worldPlane_v1, rotationMatrix.create(3,3,cv::DataType<double>::type); cv::solvePnP(worldPlane, imagePlane, intrinsics, distCoeffs, rvec, tvec); cv::Rodrigues(rvec,rotationMatrix);

cv::Rodrigues(rvec,rotationMatrix); cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type); //u,v,1

//u,v,1 // image point uvPoint.at<double>(0,0) = 3.; //got this point using mouse callback uvPoint.at<double>(1,0) = 134.;

134.; cv::Mat tempMat, tempMat2; double s, zConst = 0; tempMat = rotationMatrix.inv() * intrinsics.inv() * uvPoint; tempMat2 = rotationMatrix.inv() * tvec; s = zConst + tempMat2.at<double>(2,0); //285 represents the height Zconst s /= tempMat.at<double>(2,0); cv::Mat wcPoint = rotationMatrix.inv() * (s * intrinsics.inv() * uvPoint - tvec);

tvec); Point3f realPoint(wcPoint.at<double>(0, 0), wcPoint.at<double>(1, 0), wcPoint.at<double>(2, 0)); // point in world coordinates

coordinates

update: I made it an answer since it was not fitting in a comment, and multiple comments with code were ugly and not helping to understand

@Theoretiker you need to obtain the tvec, rvec and rotationMatrix. the first two you get them through the solvePnP() function and the latter through the Rodrigues() function. Once you have this information in addition to the camera calibration coefficients you can transform the pixel point to world coordinates point. The code should be something like that (it is taken from an old project of mine):

Mat intrinsics, distCoeffs;
rvec.create(1,3,cv::DataType<double>::type);
tvec.create(1,3,cv::DataType<double>::type);
rotationMatrix.create(3,3,cv::DataType<double>::type);

cv::solvePnP(worldPlane, imagePlane, intrinsics, distCoeffs, rvec, tvec);
cv::Rodrigues(rvec,rotationMatrix);

cv::Mat uvPoint = cv::Mat::ones(3,1,cv::DataType<double>::type); //u,v,1

// image point
uvPoint.at<double>(0,0) = 3.; //got this point using mouse callback
uvPoint.at<double>(1,0) = 134.;

cv::Mat tempMat, tempMat2;
double s, zConst = 0;
tempMat = rotationMatrix.inv() * intrinsics.inv() * uvPoint;
tempMat2 = rotationMatrix.inv() * tvec;
s = zConst + tempMat2.at<double>(2,0); //285 represents the height Zconst
tempMat2.at<double>(2,0);
s /= tempMat.at<double>(2,0);
cv::Mat wcPoint = rotationMatrix.inv() * (s * intrinsics.inv() * uvPoint - tvec);

Point3f realPoint(wcPoint.at<double>(0, 0), wcPoint.at<double>(1, 0), wcPoint.at<double>(2, 0)); // point in world coordinates