OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Fri, 14 Aug 2020 09:04:47 -0500Triangulate points from 2 images to estimate the pose on a thirdhttp://answers.opencv.org/question/233704/triangulate-points-from-2-images-to-estimate-the-pose-on-a-third/I want to estimate the pose of a current image relative to a matched one in a database. The images are coming from a moving robot with reasonably accurate odometry so I can take two consecutive images that are both similar to the one in the database and have a good estimate of the relative pose between these two images. I want to use that information to estimate the 3D relationship between the matched keypoints in the two current images using triangulatePoints and then use solvePnPRansac to estimate the pose of the image in the database relative to the first current image using the keypoints that also match to it.
If have tried to implement this in OpenCV with Python as shown below. Currently I do not use the database image and am simply confirming if I can get the odometery that I enforced back, but unfortunately the current output is garbage (translation in the order of e+10).
import numpy as np
import cv2
# Fisheye camera and distortion matrices
K=np.array([[455.5000196386718, 0.0, 482.65324003911945], [0.0, 340.6409393462825, 254.5063795692748], [0.0, 0.0, 1.0]])
D=np.array([[-0.018682808343432777], [-0.044315351694893736], [0.047678551616171246], [-0.018283908577445218]])
orb = cv2.ORB_create(nfeatures=1000)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=False)
img0 = cv2.imread("0cm.png")
img2 = cv2.imread("2cm.png")
# Find keypoints and match them up
kp0, des0 = orb.detectAndCompute(img0, None)
kp2, des2 = orb.detectAndCompute(img2, None)
matches = bf.knnMatch(des0, des2, k=2)
# Find good matches using the ratio test
ratio_thresh = 0.8
good_matches = []
for m,n in matches:
if m.distance < ratio_thresh * n.distance:
good_matches.append(m)
# Convert from keypoints to points
pts0 = np.float32([kp0[m.queryIdx].pt for m in good_matches]).reshape(-1, 1, 2)
pts2 = np.float32([kp2[m.trainIdx].pt for m in good_matches]).reshape(-1, 1, 2)
# Remove the fisheye distortion from the points
pts0 = cv2.fisheye.undistortPoints(pts0, K, D, P=K)
pts2 = cv2.fisheye.undistortPoints(pts2, K, D, P=K)
# Keep only the points that make geometric sense
# TODO: find a more efficient way to apply the mask
E, mask = cv2.findEssentialMat(pts0, pts2, K, cv2.RANSAC, 0.999, 1, None)
_, R, t, mask = cv2.recoverPose(E, pts0, pts2, cameraMatrix=K, mask=mask)
pts0_m = []
pts2_m = []
for i in range(len(mask)):
if mask[i] == 1:
pts0_m.append(pts0[i])
pts2_m.append(pts2[i])
pts0 = np.array(pts0_m).T.reshape(2, -1)
pts2 = np.array(pts2_m).T.reshape(2, -1)
# Setup the projection matrices
R = np.eye(3)
t0 = np.array([[0], [0], [0]])
t2 = np.array([[0], [0], [2]])
P0 = np.dot(K, np.concatenate((R, t0), axis=1))
P2 = np.dot(K, np.concatenate((R, t2), axis=1))
# Find the keypoint world homogeneous coordinates assuming img0 is the world origin
X = cv2.triangulatePoints(P0, P2, pts0, pts2)
# Convert from homogeneous cooridinates
X /= X[3]
objPts = X.T[:,:3]
# Find the pose of the second frame
_, rvec, tvec, inliers = cv2.solvePnPRansac(objPts, pts2.T, K, None)
print(rvec)
print(tvec)
Is there something wrong with my code or my approach (or both)?
EDIT:
I tested this with an image that is 6cm away instead of just 2cm away and it seems to work just fine then. I guess a small translation only in the forward direction is resulting in some kind of numerical instability somewhere. There is a slight difference in the results depending on the units I use to construct the projection matrices though (0.06 for metres vs 6 for centimetres), but I have no idea what units I should then actually use to get the most accurate results seeing as how it does not appear to be irrelevant. I though it could somehow be related to the camera matrix, but I obtained mine using the guide at https://medium.com/@kennethjiang/calibrate-fisheye-lens-using-opencv-333b05afa0b0 and scaling the dimensions of the checkerboard in the calibration code has no effect on the resulting matrix so now I have no idea.GerharddcFri, 14 Aug 2020 09:04:47 -0500http://answers.opencv.org/question/233704/Should I use focal length from camera matrix or new camera matrix for depth estimation?http://answers.opencv.org/question/213871/should-i-use-focal-length-from-camera-matrix-or-new-camera-matrix-for-depth-estimation/ I use initUndistortRectifyMap using camera matrix and new camera matrix to get output maps. New camera matrix was obtained using estimateNewCameraMatrixForUndistortRectify and passing a non-zero balance value. The new camera matrix as compared to camera matrix has different focal length.
Now the question is, if I get disparity values, first using remap with the output maps and then using a stereo disparity algorithm, to get the real depth what focal length value should I use? The one from old camera matrix or from the new camera matrix?KafanTue, 04 Jun 2019 07:40:45 -0500http://answers.opencv.org/question/213871/cv::undistortPoints not working for me ....http://answers.opencv.org/question/207434/cvundistortpoints-not-working-for-me/ I am trying to un-distort the pixel coordinate of two points using the code below but I am not successful.
vector<Point2f> pts;
cv::Mat upts;
pts.push_back(diagStartPnt);
pts.push_back(diagEndPnt);
cerr << "m_cameraMat = \n" << m_cameraMat << endl;
cerr << "m_distortionCoefMat = \n" << m_distortionCoefMat << endl;
cerr << "pts = \n" << pts << endl;
undistortPoints(Mat(pts), upts, m_cameraMat, m_distortionCoefMat);
cerr << "upts = " << upts << endl;
But it doesn't seem to be working, I get weird output ... !!!???
m_cameraMat =
[606.184487913021, 0, 320;
0, 606.184487913021, 240;
0, 0, 1]
m_distortionCoefMat =
[-0.02448317075341529;
0.3189340323130755;
0;
0;
-1.013832345303472]
pts =
[533.44543, 347.06061;
529.32397, 234.65332]
upts =
[0.35208049, 0.1765976;
0.34534943, -0.0088211242]
I am totally clueless ... :|mikeitexpertSat, 19 Jan 2019 02:51:40 -0600http://answers.opencv.org/question/207434/Using OpenCV to calibrate Pi Camera always gives me the same fx and fy for focal lengthhttp://answers.opencv.org/question/192028/using-opencv-to-calibrate-pi-camera-always-gives-me-the-same-fx-and-fy-for-focal-length/ Hi all,
I'm doing camera calibration using OpenCV's tutorial code from [here](https://github.com/opencv/opencv/blob/master/samples/cpp/tutorial_code/calib3d/camera_calibration/camera_calibration.cpp).
As we know, the output camera matrix should in the following matrix:<br>
[fx, 0, cx <br>
0, fy, cy <br>
0, 0, 1]
From that, the camera intrinsic matrix I got always has the same fx and fy. For example:
data:<br>[ 6.5746697810243404e+002, 0., 1560, 0.,
6.5746697810243404e+002, 1232, 0., 0., 1. ]
From the sample given by OpenCV, it also has the same fx and fy.
I don't understand why the calculated fx and fy will be the same, since the resolution for x and y is different, the sensor width and height is also different.
Is there anybody can explain this for me???
Thanks a lot.
ArlenearlenelalalaTue, 22 May 2018 01:58:08 -0500http://answers.opencv.org/question/192028/PiCamera V2, intrinsic matrix after calibration to comparehttp://answers.opencv.org/question/191543/picamera-v2-intrinsic-matrix-after-calibration-to-compare/Hi All,
I just followed the tutorial about camera calibration from OpenCV and got the camera intrinsic matrix like following:
[2581.33211, 0, 320
0, 2576, 240
0, 0, 1]
It would be really appreciated if you can share your camera matrix, I just want to compare the result.
Btw, the camera resolution should be 3280*2464, but I am not sure why my **Cx** and **Cy** is only 320 and 240.
ThanksarlenelalalaMon, 14 May 2018 17:08:56 -0500http://answers.opencv.org/question/191543/Unexpected cameraMatrix from calibrateCamerahttp://answers.opencv.org/question/191099/unexpected-cameramatrix-from-calibratecamera/When calling calibrateCamera I specify the CV_CALIB_FIX_PRINCIPAL_POINT flag. So I expect the cx and cy of the resulting cameraMatrix to reflect the middle of the screen. My camera is 1280 width and 1024 high. But I get a cx of 512 and a cy of 640. I expected the other way around because x is in the width direction.
Do I misunderstand something?IC++Tue, 08 May 2018 13:03:10 -0500http://answers.opencv.org/question/191099/2 dim matrix, how can it behttp://answers.opencv.org/question/183098/2-dim-matrix-how-can-it-be/Hello. I hope you all are fine.
Im new with this API and as every newbie I need a little help here. I'm using a banana pi m2 and a usb cam (a genius one). First of all I want to check the camera paramters, so:
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui.hpp>
using namespace cv;
using namespace std;
int main(int argc,char ** argv)
{
VideoCapture cap(0);
if (!cap.isOpened()) {
cerr << "ERROR: Unable to open the camera" << endl;
return 0;
}
Mat frame1, frame;
cout << "Start grabbing, press a key on Live window to terminate" << endl;
cap >> frame1;
if (frame1.empty())
{
cerr << "ERROR: Unable to grab from the"<<endl;
}
int i, j, k,z;
i=frame1.dims;
j=frame1.cols;
k=frame1.rows;
z=frame1.type();
printf("%d dim\n %d col \n %d row\n %d type\n", i, j, k, z);
return 1;
}
And this is what a I got:
Start grabbing, press a key on Live window to terminate
2 dim
640 col
480 row
16 type
How can be a matrix 2 dimension??? So, thats means that it has just 2 colors? its a two channel?
Also, a 16 type means CV_8U. So there is 8 bits per color(R G or B)? because VGA use 6bits per color
Thanks in advance.
julian403Tue, 23 Jan 2018 18:16:25 -0600http://answers.opencv.org/question/183098/3D reconstruction (SfM) - confusion with camera's extrinsic parameterhttp://answers.opencv.org/question/119359/3d-reconstruction-sfm-confusion-with-cameras-extrinsic-parameter/From two images (with known camera intrinsic parameter), I follow the usual pipeline to reconstruct its 3D points. In particular, I use the `cv2.findEssentialMat` and `cv2.recoverPose` function. From the `cv2.recoverPose`, I got the rotation and translation of the second camera (the first camera will has identity matrix for its rotation and zero vector for its translation). What confuse me is the value of the camera 2's translation that is `[ -0.98681175 0.08603786 0.1371133 ]`, where I got a negative x value. Clearly from my images, the second camera should move right, indicating a positive x value. Why is this? Is it because the matrix should show the movement of the points and not the camera itself (I always thought the extrinsic parameter shows the movement of the camera)?
Below are the two images (with highlighted keypoints used for 3D reconstruction) and the reconstructed scene.
![image description](/upfiles/148188731320673.jpg)
![Reconstructed scene](/upfiles/1481886957445836.png)
The two blue X marks above are camera 1 and camera 2 position (here, I have multiplied camera 2's translation with -1 to get a positive x value).HilmanFri, 16 Dec 2016 05:33:19 -0600http://answers.opencv.org/question/119359/What algorithm does cv::findChessboardCorners use?http://answers.opencv.org/question/88368/what-algorithm-does-cvfindchessboardcorners-use/ Hi,
I wonder what algorithm is used in cv::findChessboardCorners. I found explanations for a lot of funcrions in openCV but no information about cv::findChessboardCorners. Is it something like a Harris coner detector or some other standard algorithm?
~TankardTankardTue, 23 Feb 2016 07:12:23 -0600http://answers.opencv.org/question/88368/