OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Thu, 17 Jan 2019 11:24:49 -0600Transform 2D Point into 3D Linehttp://answers.opencv.org/question/207274/transform-2d-point-into-3d-line/Im looking for a function that transforms a 2D image-point into a 3D line in my model (specific coord system, belonging to translation and rotation vectors received by function solvePnP() ).
I have: cameraMatrix, rotation and translation vectors, distortionCoefficients.
It would be the inverse function of projectPoints(), which takes a 3D point and transforms it into a 2d image point.
Is there any solution for this issue?
LuisKThu, 17 Jan 2019 11:24:49 -0600http://answers.opencv.org/question/207274/How to find coordinates of screen corner if camera mounted on screenhttp://answers.opencv.org/question/203777/how-to-find-coordinates-of-screen-corner-if-camera-mounted-on-screen/ Hello,
My setup is a typical laptop with a built in webcam.
I'm trying to find intersection of a vector seen by camera with my laptop screen.
I know the vector equation, I have camera calibrated.
However, I now need to know the equation of plane in which the screen lies. For this, I need camera coordinates of any 3 points on the screen.
Since camera is in the same plane as the screen, it can't "see" the screen, how do I estimate the top left corner of screen if I know that it is 17.5 cm to the left of the camera?
Is there a way to convert physical distance into camera coordinates?
I'm assuming in camera coordinates, the camera centre itself is (0,0,0) and therefore the left screen corner would be (-17.5cm,-1cm,0) - Z remaining 0 since its in same plane, & X,Y values being negative since its below to the left of camera. But how to get this in camera coordinates?
VBWed, 21 Nov 2018 06:39:04 -0600http://answers.opencv.org/question/203777/Is cv::triangulatePoints() returning 3D points in world coordinate system?http://answers.opencv.org/question/118966/is-cvtriangulatepoints-returning-3d-points-in-world-coordinate-system/Considering a moving camera with fixed calibration matrix (intrinsic parameters), I am triangulating tracked feature points from two views that are not consecutive. The view poses are in camera coordinate system and images are already undistorted before detecting and tracking features.
Please can you confirm if the triangulated points are in world coordinate system after applying the cv::triangulatePoints() and cv::convertPointsFromHomogeneous() functions.ale_XompiWed, 14 Dec 2016 08:54:28 -0600http://answers.opencv.org/question/118966/Convert Pose from ARToolkit into OpenCV Screen Coordinateshttp://answers.opencv.org/question/182075/convert-pose-from-artoolkit-into-opencv-screen-coordinates/I have modified the ARToolkit **nftSimple** Application to work **with OpenCV Mat** and **without OpenGL** dependencies. I have got the tracking to work successfully and i am able to extract the pose matrix **ARPose**.
**Code for project :- https://github.com/njanirudh/OpenCV-ARTK**
## Output
The ARPose is of the format (16 x 1) :
_Documentation : // Position and orientation, column-major order. (position(x,y,z) = {T[12], T[13], T[14]}_
**T** = 0.833453 0.148213 0.532342 0.000000 -0.055585 0.980959 -0.186091 0.000000
-0.549787 0.125508 0.825822 0.000000 60.233962 -307.641967 -964.580499 1.000000
## Problem
I am trying to use this to draw a pink circle on OpenCV side using the x,y,z values returned from the pose.
According to the documentation in the code
T[12] = 60.233962 , T[13] = -307.641967 , T[14] = -964.580499
![Screenshot from the Application](https://user-images.githubusercontent.com/15691827/34757378-bf2c2136-f5f6-11e7-9f06-fc59c97704bf.png?raw=true)
The coordinates obtained from the pose matrix are not giving the correct value.
**How do we transform the pose matrix from ARToolkit into OpenCV Screen coordinates?**
static void runDetection(Mat &input_mat)
{
ARUint8 *gARTImage = NULL;
gARTImage = (uchar *)input_mat.data;
float trackingTrans[3][4] = {0,0,0,0,0,0,0,0,0,0,0,0};
cv::Mat rot_vec = Mat::zeros(1,3,CV_64F), trn_vec = Mat::zeros(1,3,CV_64F) , rot_mat = Mat::zeros(3,3,CV_64F);
if(true){
// NFT results.
static int detectedPage = -2; // -2 Tracking not inited, -1 tracking inited OK, >= 0 tracking online on page.
int i, j, k;
// Grab a video frame.
if (gARTImage != NULL) {
gCallCountMarkerDetect++; // Increment ARToolKit FPS counter.
// Run marker detection on frame
if (threadHandle) {
// Perform NFT tracking.
float err;
int ret;
int pageNo;
if( detectedPage == -2 ) {
trackingInitStart( threadHandle, gARTImage );
detectedPage = -1;
}
if( detectedPage == -1 ) {
ret = trackingInitGetResult( threadHandle, trackingTrans, &pageNo);
if( ret == 1 ) {
if (pageNo >= 0 && pageNo < surfaceSetCount) {
ARLOGd("Detected page %d.\n", pageNo);
detectedPage = pageNo;
ar2SetInitTrans(surfaceSet[detectedPage], trackingTrans);
} else {
ARLOGe("Detected bad page %d.\n", pageNo);
detectedPage = -2;
}
} else if( ret < 0 ) {
ARLOGd("No page detected.\n");
detectedPage = -2;
}
}
if( detectedPage >= 0 && detectedPage < surfaceSetCount) {
if( ar2Tracking(ar2Handle, surfaceSet[detectedPage], gARTImage, trackingTrans, &err) < 0 ) {
ARLOGd("Tracking lost.\n");
detectedPage = -2;
} else {
ARLOGd("Tracked page %d (max %d).\n", detectedPage, surfaceSetCount - 1);
}
}
} else {
ARLOGe("Error: threadHandle\n");
detectedPage = -2;
}
// Update markers.
for (i = 0; i < markersNFTCount; i++) {
markersNFT[i].validPrev = markersNFT[i].valid;
if (markersNFT[i].pageNo >= 0 && markersNFT[i].pageNo == detectedPage) {
markersNFT[i].valid = TRUE;
for (j = 0; j < 3; j++)
for (k = 0; k < 4; k++){
markersNFT[i].trans[j][k] = trackingTrans[j][k];
}
}
else markersNFT[i].valid = FALSE;
if (markersNFT[i].valid) {
// Filter the pose estimate.
if (markersNFT[i].ftmi) {
if (arFilterTransMat(markersNFT[i].ftmi, markersNFT[i].trans, !markersNFT[i].validPrev) < 0) {
ARLOGe("arFilterTransMat error with marker %d.\n", i);
}
}
if (!markersNFT[i].validPrev) {
// Marker has become visible, tell any dependent objects.
// --->
}
// We have a new pose, so set that.
arglCameraViewRH((const ARdouble (*)[4])markersNFT[i].trans, markersNFT[i].pose.T, VIEW_SCALEFACTOR);
//Print(markersNFT[i].pose.T);
//************Rotation Matrix************
rot_mat.at<float>(0,0) = trackingTrans[0][0];
rot_mat.at<float>(0,1) = trackingTrans[0][1];
rot_mat.at<float>(0,2) = trackingTrans[0][2];
rot_mat.at<float>(1,0) = trackingTrans[1][0];
rot_mat.at<float>(1,1)= trackingTrans[1][1];
rot_mat.at<float>(1,2) = trackingTrans[1][2];
rot_mat.at<float>(2,0) = trackingTrans[2][0];
rot_mat.at<float>(2,1) = trackingTrans[2][1];
rot_mat.at<float>(2,2) = trackingTrans[2][2];
Rodrigues(rot_mat, rot_vec);
//************Translation Matrix***********
trn_vec.at<float>(0,0) = markersNFT[i].pose.T[12];
trn_vec.at<float>(0,1) = markersNFT[i].pose.T[13];
trn_vec.at<float>(0,2) = markersNFT[i].pose.T[14];
//************Camera Matrix*****************
cv::Mat intrisicMat(3, 3, cv::DataType<double>::type); // Intrisic matrix
intrisicMat.at<double>(0, 0) = 674.171631;
intrisicMat.at<double>(1, 0) = 0;
intrisicMat.at<double>(2, 0) = 0;
intrisicMat.at<double>(0, 1) = 0;
intrisicMat.at<double>(1, 1) = 633.898087;
intrisicMat.at<double>(2, 1) = 0;
intrisicMat.at<double>(0, 2) = 318.297791;
intrisicMat.at<double>(1, 2) = 237.900467;
intrisicMat.at<double>(2, 2) = 1;
//************Dist Matrix**********************
cv::Mat distCoeffs(5, 1, cv::DataType<double>::type); // Distortion vector
distCoeffs.at<double>(0) = 0;
distCoeffs.at<double>(1) = 0;
distCoeffs.at<double>(2) = 0;
distCoeffs.at<double>(3) = 0;
distCoeffs.at<double>(4) = 0;
std::vector<cv::Point3d> inputPnt ;
std::vector<cv::Point2d> outPnt;
cv::Point3d pnt = Point3d(markersNFT[i].pose.T[12],markersNFT[i].pose.T[13],markersNFT[i].pose.T[14]);
inputPnt.push_back(pnt);
cv::projectPoints(inputPnt,rot_vec,trn_vec,intrisicMat,distCoeffs,outPnt);
// Convert from ARToolkit Image type to OpenCV Mat
IplImage* iplImg;
iplImg = cvCreateImage(cvSize(640, 480), IPL_DEPTH_8U,3);
// Get video frame from ARToolkit
iplImg->imageData = (char *)gARTImage;
input_mat = cv::cvarrToMat(iplImg);
cout<<outPnt<<"**********"<<endl;
//****Draw a Circle using the pose data****
cv::circle(input_mat,Point(outPnt[0].y,0),30,Scalar(255,255,255) ,4);
} else {
if (markersNFT[i].validPrev) {
// Marker has ceased to be visible, tell any dependent objects.
// --->
}
}
}
}
}
}njTue, 09 Jan 2018 23:55:03 -0600http://answers.opencv.org/question/182075/Transform world coordinate to Camera Coordinate using SolvePNP?http://answers.opencv.org/question/173614/transform-world-coordinate-to-camera-coordinate-using-solvepnp/ I estimate rvec and tvec using solvePNP . How can I convert the (0,0,0) point from world coordinate to the camera coordinate using rvec and tvec? Our model points is shown below:
![image description](/upfiles/1504601071823469.jpg)
ibrahim_5403Tue, 05 Sep 2017 03:48:32 -0500http://answers.opencv.org/question/173614/Real World Coordinate of Rotary table in Stereo Calibrationhttp://answers.opencv.org/question/162222/real-world-coordinate-of-rotary-table-in-stereo-calibration/ I have implemented stereo camera calibration and I can get the extrinsic and intrinsic camera parameter. For calibration I have captured the calibration board with 6-DOF. But now calibration board is mounted on the rotary table and I need to calculate the position (XZ) of rotary table respect to the origin point which is camera 0.
![image description](https://i.stack.imgur.com/EHk0g.png)
My idea to get the XZ position of rotary table is:
1- capturing some calibration board images by rotating the table 2- calculating the normal vector of calibration board 3- finding the cross sections of those normal vectors (at least two)
My question is how can I calculate the normal vector of calibration board? or maybe someone has better solution to get the world coordinate of the rotary table.
I googled it but couldn't find something relevant.AlexxpSun, 25 Jun 2017 07:53:14 -0500http://answers.opencv.org/question/162222/Generate different camera view using camera poseshttp://answers.opencv.org/question/129367/generate-different-camera-view-using-camera-poses/
I have obtained multiple views of a scene using different poses of the same camera in Blender. The poses of cameras are given by `(tx, ty, tz)` and 4-tuple quaternion `(w, x, y, z)` with respect to the world coordinate system. Now, given I have the poses of all the camera views, I want to generate the camera view of a camera `C2` with rotation and translation as `R2` and `T2`, given I know the pose of another camera `C1` with `R1` and `T1`. Then I want to compare how close the generated image is to the groundtruth image (as generated by Blender). For generating the image, I use a grid of points and warp the grid using the transformation matrix. Then I use GPU to warp the image. Let's consider the reference camera's grid points be `p1`. The way I find the warped points is by doing the following:
p2 = K*inv(R2)*R1*p1 + K*inv(R2)*(T1 - T2)
where `K` is the intrinsic parameters.
My question is: Is the generated image going to be exactly same as the groundtruth image ? Because only the distant features of the image are aligning with the groundtruth. The nearby objects are not aligned properly. Is this because of parallax as the optical centers are not same for both the cameras ?bluechillkkiTue, 21 Feb 2017 02:12:39 -0600http://answers.opencv.org/question/129367/Interpret results from triangulatePointshttp://answers.opencv.org/question/104557/interpret-results-from-triangulatepoints/Hi,
I am detecting ArUco Markers with a stereo setup and would like to know the 3d coordinates of the corners. I am using triangulatePoints to achieve that (the rig is fully calibrated and I am calling undistortPoints before triangulating), but I do not understand how to interpret the results. Here is an example:
![left image](/upfiles/14768718144111549.png)
![right image](/upfiles/14768718303093121.png)
As you can see the markers are detected fine. The results of triangulatePoints are the following:
0.247877 0.0300715 0.501093
0.254448 0.0923606 0.518614
0.181621 0.0959466 0.508083
0.176167 0.0358917 0.50486
0.00881887 0.0501222 0.502481
0.00898725 0.00313973 0.520062
0.0636986 -0.00419561 0.526967
0.0654933 0.0450242 0.509843
0.166304 -0.163573 0.579394
0.225936 -0.172218 0.58141
0.230371 -0.112224 0.581557
0.170264 -0.104482 0.576754
0.0295858 -0.132247 0.574503
0.0318483 -0.16779 0.591691
0.0783909 -0.178207 0.602229
0.0787127 -0.13994 0.583072
0.151794 -0.236149 0.629165
0.102628 -0.229509 0.624732
0.0989285 -0.286064 0.634169
0.151599 -0.29231 0.637837
I tried to plot these in Matlab, and the result looks good:
![Matlab plot](/upfiles/1476872189605449.png)
But I just cannot find any relations to the cameras position. I would like to have coordinates somehow relative to one of the cameras, so that the camera is the origin of the used coordinate system. How can I do that? I read that the origin is the optical center of the first camera, but then my x-values should not be all-positive, right?benTThu, 13 Oct 2016 09:10:36 -0500http://answers.opencv.org/question/104557/Where to set origin of model for solvePnPhttp://answers.opencv.org/question/100840/where-to-set-origin-of-model-for-solvepnp/Hi everyone, I've been figuratively tearing my hair out over this problem for over a month now. I have to admit that I haven't taken a good hard look at the math behind solvePnP (most people don't, which is why we have open source libraries), because reasons.
I'd like to ask for some practical advice regarding using solvePnP with regards to where to position the origin for the model.
Background: I'm using OpenCV to make a head tracking app. Tracking the objects (4 of them) on screen isn't really the problem; figuring out the 6DoF is.
Camera: Mobile phone camera. I haven't performed calibration, but I figure that as long as the subject keeps their head in the center of the view, it shouldn't be a big problem. I do notice significant barrel distortion towards the edges though.
Parameters fed into solvePnP: Image points -- from tracker (origin in lower-left). Model points -- from 3D model (origin in lower-left, translated to OpenCV upper-left origin). Camera matrix -- fx,fy max dimension of image, principal point at center of image, no skew (may change if I do calibration). Using ITERATIVE with extrinsicGuess (because it's very jittery without it).
So the specific advice that I want to ask: Where should I set the origin of the model (in this case, of a human head) to get consistent 6DoF values? Ideally, I want to get the rotation values "anchored" inside the neck (about a few centimeters deep from the anterior). I've tried translating the head model points to have the origin inside the neck, but I notice that z-translation values go bonkers when a significant portion of the model is in the -z range. Also, it becomes unable to resolve x-rotation properly (it gives me positive values whether I look up or down).
Should I be placing the model in a particular octant of the model space? Should I avoid having any part of my model in the -z range, or should I expect solvePnP to return a -z translation for such a situation? Melvin_DYMon, 22 Aug 2016 19:29:14 -0500http://answers.opencv.org/question/100840/Triangulation origin with stereo systemhttp://answers.opencv.org/question/90124/triangulation-origin-with-stereo-system/I m using a stereo system and so i m trying to get world coordinates of some points.
I can do it with specific calibration for each camera and then i calculate rotation matrix and
translation vector. And finally i triangulate but i m not sure of the origin of the world coordinates.
As you can see on my figure, values correspond to depth value but they shoud be close of 400 as it is flat.
So i suppose that the origin is the left camera that s why it variates...
![image description](/upfiles/14580381341122406.png)
A piece of my code with my projective arrays and triangulate function :
#C1 and C2 are the cameras matrix (left and rig)
#R_0 and T_0 are the transformation between cameras
#Coord1 and Coord2 are the correspondant coordinates of left and right respectively
P1 = np.dot(C1,np.hstack((np.identity(3),np.zeros((3,1)))))
P2 =np.dot(C2,np.hstack(((R_0),T_0)))
for i in range(Coord1.shape[0])
z = cv2.triangulatePoints(P1, P2, Coord1[i,],Coord2[i,])
My cameras present an angle, the Z axis direction (direction of the depth) is not normal to my surface. And i want the depth from the baseline direction. So i have to rotate my points?
![image description](/upfiles/14580784518525808.png)
Thanks for help ;)Bilou563Tue, 15 Mar 2016 05:41:44 -0500http://answers.opencv.org/question/90124/Rotation center confusionhttp://answers.opencv.org/question/82708/rotation-center-confusion/Can someone please verify my understanding or correct my misunderstanding?
If I rotate an image using any one of the many online examples (mainly Stackoverflow), I get an unexpected offset in the rotated image. The examples I've found online all use center=(cols/2.0, rows/2.0) for the affine transform getRotationMatrix2D() call. I believe this is incorrect. I think this should instead be center=(cols/2.0 - 0.5, rows/2.0 - 0.5). Since the online concensus seems against me, I'm really just hoping someone here will verify that I'm not incorrect in my understanding.
Specifically, in Python, I need to do this to get what I believe is correct behavior:
def rotate_opencv(image, angle):
'''From http://stackoverflow.com/questions/9041681/opencv-python-rotate-image-by-x-degrees-around-specific-point
'''
image_center = tuple(np.array(image.shape)/2.0-0.5)
# Note: -0.5 offset; see http://answers.opencv.org/question/35111/origin-pixel-in-the-image-coordinate-system-in-opencv/
rot_mat = cv2.getRotationMatrix2D(image_center, angle, 1.0)
result = cv2.warpAffine(image, rot_mat, image.shape, flags=cv2.INTER_LINEAR)
return result
I verified this by comparing the result of rotation by 90 degrees with a direct transpose-and-flip-based rotation (numpy.rot90) function call.
I think it is related to this:
http://answers.opencv.org/question/33516/cv2warpaffine-results-in-an-image-shifted-by-05-pixel/
If I understand that correctly, the pixel-coordinate-system and affine-transform-coordinate-system are offset by 0.5 pixels, leading to the common error.grubenThu, 07 Jan 2016 19:46:50 -0600http://answers.opencv.org/question/82708/Coordinate axis with triangulatePointshttp://answers.opencv.org/question/71653/coordinate-axis-with-triangulatepoints/So, I have the projection matrix of the left camera:
![image description](/upfiles/14431709882413459.png)
and the projection matrix of my right camera:
![image description](/upfiles/14431710436527588.png)
And when I perform `triangulatePoints` on the two vectors of corresponding points, I get the collection of points in 3D space.
All of the points in 3D space have a **negative Z coordinate**. I **assume** that the initial orientation of each camera is directed **in the positive Z axis direction**.
My assumption was that OpenCV uses Right Hand coordinate system like this:
![image description](/upfiles/14431714989453053.png)
![image description](https://upload.wikimedia.org/wikipedia/commons/b/b2/3D_Cartesian_Coodinate_Handedness.jpg)
So, when I positioned my cameras with projection matrices, the complete picture would look like this:
![image description](/upfiles/14431719796004324.png)
But my experiment leads me to believe that OpenCV uses Left Hand coordinate system:
![image description](/upfiles/14431715414653933.png)
And that my projection matrices have effectively messed up the left and right concept:
![image description](/upfiles/14431720286658647.png)
Is everything I've said correct? Is the latter coordinate system really the one that is used by OpenCV?
If I assume that it is, everything seems to work fine. But when I want to visualize things using `viz` module, their `WCoordinateSystem` widget is right handed.acajicFri, 25 Sep 2015 04:11:22 -0500http://answers.opencv.org/question/71653/