OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Fri, 18 Dec 2020 07:38:58 -0600Problem with undistortPoints() function in pose estimation of imagehttp://answers.opencv.org/question/239443/problem-with-undistortpoints-function-in-pose-estimation-of-image/ I have written about my task [here](https://answers.opencv.org/question/238792/problem-with-building-pose-mat-from-rotation-and-translation-matrices/). I have
a set of images with known pose which were used for scene reconstruction and some query image from the same space without pose. I need to calculate the pose of the query image. I solved this problem using essential matrix. Here is a code
Mat E = findEssentialMat(pts1, pts2, focal, pp, FM_RANSAC, F_DIST, F_CONF, mask);
// Read pose for view image
Mat R, t; //, mask;
recoverPose(E, pts1, pts2, R, t, focal, pp, mask);
The only problem is that OpenCv documentation [states](https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html#ga13f7e34de8fa516a686a56af1196247f) that the findEssentialMat function assumes that points1 and points2 are feature points from cameras with the same camera intrinsic matrix. That's not a case for us - images of scene and query image can be captured by cameras with different intrinsics.
I suppose to use this undistortPoints() function. According to [documentation](https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga55c716492470bfe86b0ee9bf3a1f0f7e) the undistortPoints() function takes two important parameters distCoeffs and cameraMatrix.
Both images of the scene and query image have calibration parameters associated (fx, fy, cx, cy).
I obtain cameraMatrix parameter this way:
Mat K_v = (Mat_<double>(3, 3) <<
fx, 0, cx,
0, fy, cy,
0, 0, 1, CV_64F);
Is this correct? Moreover I need to get somewhere distCoeffs. So how I can obtain this distortion coefficients for images of the scene and query image? Probably I should solve it another way?sigmoid90Fri, 18 Dec 2020 07:38:58 -0600http://answers.opencv.org/question/239443/Triangulation with Ground Plane as Originhttp://answers.opencv.org/question/238744/triangulation-with-ground-plane-as-origin/Hello, I am working on a project where I have two calibrated cameras (c1, c2) mounted to the ceiling of my lab and I want to triangulate points on objects that I place in the capture volume. I want my final output 3D points to be relative to the world origin that is placed on the ground plane (floor of the lab). I have some questions about my process and multiplying the necessary transformations. Here is what I have done so far...
To start I have captured an image, with c1, of the ChArUco board on the ground that will act as the origin of my "world". I detect corners ( cv::aruco::detectMarkers/ cv::aruco::interpolateCornersCharuco) in the image taken by c1 and obtain the transformation (with cv::projectPoints) from 3D world coordinates to 3D camera coordinates.
![transform of board coords to camera 1 coords](https://latex.codecogs.com/gif.latex?%5Cbegin%7Bpmatrix%7D%7BX_c_1%7D%20%5C%5C%20%7BY_c_1%7D%20%5C%5C%20%7BZ_c_1%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D%20%3D%20%5E%7Bc1%7DM_%7Bboard%7D%20%5Cbegin%7Bpmatrix%7D%7BX_%7Bboard%7D%7D%20%5C%5C%20%7BY_%7Bboard%7D%7D%20%5C%5C%20%7BZ_%7Bboard%7D%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D)
I followed the same process of detecting corners on the ChArUco board with c2 (board in same position) and obtained the transformation that takes a point relative to the board origin to the camera origin...
![transform of board coords to camera 2 coords](https://latex.codecogs.com/gif.latex?%5Cbegin%7Bpmatrix%7D%7BX_c_2%7D%20%5C%5C%20%7BY_c_2%7D%20%5C%5C%20%7BZ_c_2%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D%20%3D%20%5E%7Bc2%7DM_%7Bboard%7D%20%5Cbegin%7Bpmatrix%7D%7BX_%7Bboard%7D%7D%20%5C%5C%20%7BY_%7Bboard%7D%7D%20%5C%5C%20%7BZ_%7Bboard%7D%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D)
**Q1. With the two transformations, and my calibrated intrinsic parameters, should I be able to pass these to cv::triangulatePoints to obtain 3D points that are relative to the ChArUco board origin?**
Next, I was curious if I use cv::stereoCalibrate with my camera pair to obtain the transformation from camera 2 relative points to camera 1 relative points, could I combine this with the transform from camera 1 relative points to board relative points...to get a transform from camera 2 relative points to board relative points...
After running cv::stereoCalibrate I obtain (where c1 is the origin camera that c2 transforms to)...
![transform of camera 2 coords to camera 1 coords](https://latex.codecogs.com/gif.latex?%5Cbegin%7Bpmatrix%7D%7BX_c_1%7D%20%5C%5C%20%7BY_c_1%7D%20%5C%5C%20%7BZ_c_1%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D%20%3D%20%5E%7Bc1%7DM_%7Bc2%7D%20%5Cbegin%7Bpmatrix%7D%7BX_%7Bc2%7D%7D%20%5C%5C%20%7BY_%7Bc2%7D%7D%20%5C%5C%20%7BZ_%7Bc2%7D%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D)
**Q2. Should I be able to combine transforms in the follow manner to get a transform that is the same (or very close) as my transform for board points to camera 2 points?**
![combined transforms](https://latex.codecogs.com/gif.latex?%5Cbegin%7Bpmatrix%7D%7BX_c_2%7D%20%5C%5C%20%7BY_c_2%7D%20%5C%5C%20%7BZ_c_2%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D%20%3D%20%28%5E%7Bc1%7DM_%7Bc2%7D%29%5E%7B-1%7D%20%5Ccdot%20%5E%7Bc1%7DM_%7Bboard%7D%20%5Cbegin%7Bpmatrix%7D%7BX_%7Bboard%7D%7D%20%5C%5C%20%7BY_%7Bboard%7D%7D%20%5C%5C%20%7BZ_%7Bboard%7D%7D%20%5C%5C%201%20%5Cend%7Bpmatrix%7D)
![combined transforms approximation](https://latex.codecogs.com/gif.latex?%5E%7Bc2%7DM_%7Bboard%7D%20%5Capprox%20%28%5E%7Bc1%7DM_%7Bc2%7D%29%5E%7B-1%7D%20%5Ccdot%20%5E%7Bc1%7DM_%7Bboard%7D)
**I tried to do this and noticed that the transform obtained by detecting the ChArUco board corners is significantly different than the one obtained by combing the transformations. Should this work as I stated, or have I misunderstood something and done the math incorrectly? Here is output I get for the two methods (translation units are meters)...**
Output from projectPoints
![](https://latex.codecogs.com/gif.latex?%5E%7Bc2%7DM_%7Bboard%7D%20%3D%20%5Cbegin%7Bbmatrix%7D%200.9844968%20%26%20-0.14832049%20%26%200.09363274%20%26%20-0.7521725%5C%5C%200.01426749%20%26%20-0.46433134%20%26%20-0.88554664%20%26%201.10571043%20%5C%5C%200.17482132%20%26%200.87315373%20%26%20-0.45501656%20%26%203.89971067%20%5C%5C%200%20%26%200%20%26%200%20%26%201%20%5Cend%7Bbmatrix%7D)
Output from combined transforms (projectPoints w/ c1 and board, and stereoCalibrate w/ c1 and c2)
![](https://latex.codecogs.com/gif.latex?%28%5E%7Bc1%7DM_%7Bc2%7D%29%5E%7B-1%7D%20%5Ccdot%20%5E%7Bc1%7DM_%7Bboard%7D%20%3D%20%5Cbegin%7Bbmatrix%7D%200.9621638%20%26%20-0.00173254%20%26%200.01675597%20%26%20-1.03920386%5C%5C%20-0.00161398%20%26%20-0.51909025%20%26%20-0.06325754%20%26%200.02077932%20%5C%5C%20-0.01954778%20%26%20-0.07318432%20%26%20-0.49902605%20%26%201.0988982%20%5C%5C%200%20%26%200%20%26%200%20%26%201%20%5Cend%7Bbmatrix%7D)
Looking at the transform obtained from projectPoints the translation makes sense as in the physical setup the ChArUco board is about 4m away from the camera. This makes me think the combined transform doesn't really make sense...
edit/update: Adding raw data from projectPoints and stereoCalibrate:
Sorry for the delay. Going through my code I use estimatePoseCharucoBoard to get my transformation matric from board coords to camera, sorry about that! Here are the matrices that I obtained;
**Note: Any time that a calibration board object is needed the board dimensions given are in meters. So scaling should remain the same between matrices.**
board to camera 190 from estimatePoseCharucoBoard -->
c1^M_board
[[ 0.99662517 0.05033606 -0.06484257 -0.88300593]
[-0.02915834 -0.52132771 -0.85285826 0.82721859]
[-0.07673376 0.85187071 -0.5181006 4.03620873]
[ 0. 0. 0. 1. ]]
board to camera 229 from estimatePoseCharucoBoard -->
c2^M_board
[[ 0.9844968 -0.14832049 0.09363274 -0.7521725 ]
[ 0.01426749 -0.46433134 -0.88554664 1.10571043]
[ 0.17482132 0.87315373 -0.45501656 3.89971067]
[ 0. 0. 0. 1. ]]
camera 229 to camera 190 from stereoCalibrate -->
c1^M_c2
[[ 0.96542194 0.05535236 0.2547481 -1.20694685]
[-0.03441951 0.99570816 -0.08591013 0.03888629]
[-0.25841009 0.07417122 0.96318371 0.04002158]
[ 0. 0. 0. 1. ]]
Here is a code snippet showing how I obtain the transformation matrix to ground:
// detect markers
aruco::detectMarkers(image, dictionary, corners, ids, detectorParams, rejected);
// Attempt to refind more markers based on already detected markers
aruco::refineDetectedMarkers(image, charucoboard, corners, ids, rejected,
noArray(), noArray(), 10.f, 3.f, true, noArray(), detectorParams);
if(ids.size() < 1){
cerr << "No marker ID's found" << endl;
}
// interpolate charuco corners
Mat currentCharucoCorners, currentCharucoIds;
aruco::interpolateCornersCharuco(corners, ids, image, charucoboard,
currentCharucoCorners, currentCharucoIds);
if(currentCharucoCorners.rows < 6) {
cerr << "Not enough corners for calibration" << endl;
}
cout << "Corners Found: " << currentCharucoCorners.rows << endl;
cout << "Total Object Points: " << objPoints.size() << endl;
aruco::estimatePoseCharucoBoard(currentCharucoCorners, currentCharucoIds, charucoboard,
intrinsics.cameraMatrix, intrinsics.distCoeffs,
rvec, tvec, false);
Rodrigues(rvec, R);
cout << "Rotation Matrix: " << R << endl;
cout << "Translation Matrix: " << tvec << endl;
P = RTtoP(R, tvec);
cout << "Projection Matrix: " << P << endl;ConnorMFri, 04 Dec 2020 10:49:29 -0600http://answers.opencv.org/question/238744/Problem with building pose Mat from rotation and translation matriceshttp://answers.opencv.org/question/238792/problem-with-building-pose-mat-from-rotation-and-translation-matrices/I have two images captured in the same space (scene), one with known pose. I need to calculate the pose of the second (query) image. I have obtained the relative camera pose using essential matrix. Now I am doing calculation of camera pose through the matrix multiplication ([here](https://answers.opencv.org/question/31421/opencv-3-essentialmatrix-and-recoverpose/) is a formula).
I try to build the 4x4 pose Mat from rotation and translation matrices. My code is following
Pose bestPose = poses[best_view_index];
Mat cameraMotionMat = bestPose.buildPoseMat();
cout << "cameraMotionMat: " << cameraMotionMat.rows << ", " << cameraMotionMat.cols << endl;
float row_a[4] = {0.0, 0.0, 0.0, 1.0};
Mat row = Mat::zeros(1, 4, CV_64F);
cout << row.type() << endl;
cameraMotionMat.push_back(row);
// cameraMotionMat.at<float>(3, 3) = 1.0;
Earlier in code. Fora each view image:
Mat E = findEssentialMat(pts1, pts2, focal, pp, FM_RANSAC, F_DIST, F_CONF, mask);
// Read pose for view image
Mat R, t; //, mask;
recoverPose(E, pts1, pts2, R, t, focal, pp, mask);
Pose pose (R, t);
poses.push_back(pose);
Initially method bestPose.buildPoseMat() returns Mat of size (3, 4). I need to extend the Mat to size (4, 4) with row [0.0, 0.0, 0.0, 1.0] (zeros vector with 1 on the last position). Strangely I get following output when print out the resultant matrix
> [0.9107258520121255,
> 0.4129580377861768, 0.006639390377046724, 0.9039011699443721;
> 0.4129661348384583, -0.9107463665340377, 0.0001652925667582038, -0.4277340727282191;
> 0.006115059555925467, 0.002591307168000504, -0.9999779453436902, 0.002497598952195387;
> 0, 0.0078125, 0, 0]
Last row does not look like it should be: [0, 0.0078125, 0, 0] rather than [0.0, 0.0, 0.0, 1.0]. Is this implementation correct? What could a problems with this matrix?sigmoid90Sun, 06 Dec 2020 04:51:36 -0600http://answers.opencv.org/question/238792/Determining types of 3D transformation between two sets of cloud pointshttp://answers.opencv.org/question/226378/determining-types-of-3d-transformation-between-two-sets-of-cloud-points/ Hi. I am new in the forum and I have a little question.
I have two cloud of points in 3D and I know that the second cloud is obtained applying a transformation matrix.
I think, if I am not mistaken, that I need to use the [estimateAffine3D](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html?highlight=estimate#cv2.estimateAffine3D) function.
But, in this case, I would like to know how much of each transformation has been applied. For example, I would like to know if the second set of clouds has been scaled 2 times, rotated 25º and sheared 5 units in the X axis.
The second set of points can have any kind of transformations.
thanks a lot for your time!
IbaiSun, 16 Feb 2020 18:03:37 -0600http://answers.opencv.org/question/226378/Refining perspective transformation in epipolar geometryhttp://answers.opencv.org/question/7689/refining-perspective-transformation-in-epipolar-geometry/Given two epipolar pics of the same scene from (slightly) different angles, and given corresponding points in the two pics that were manually curated (golden truth), one can calculate the perspective transformation matrix between the two camera planes.
However, when applying this transformation on a new point in one of the images to calculate the location of the corresponding coordinates of the point in the other image, often time the calculated coordinates hae a (small) offset from the real location (resulting from noise, approximation, etc).
What can be done to improve the accuracy of the location of the calculated point? What is the state of the art in this regard? Are there any post-refinements based on local descriptors and local search that could help improve the accuracy of the location?cv techTue, 19 Feb 2013 10:22:51 -0600http://answers.opencv.org/question/7689/Perspective transformation - Deriving formula for single Camerahttp://answers.opencv.org/question/221482/perspective-transformation-deriving-formula-for-single-camera/I would like to solve an equation and of that formulate a formula that takes two inputs and gives one output:
**Input:**
- (u,v) - Pixel coordinates
- (t) - Translation of the camera with respect to the plane, in one dimension (z)
**Output**
- (x,y) - World coordinate
And for the rotation of the camera, this is set to static. So the only part that can alternate is the height of the camera with respect to the plane.
I've successfully solved an equation system, but for when the camera has fixed rotation and fixed height as described here: https://dsp.stackexchange.com/a/46591/46122
But now I want to express a formula but that takes one additional parameter (height in [mm]).
However, I'm not sure how this equation system would look like described here, https://dsp.stackexchange.com/a/46591/46122
to reflect my additional parameter.
My goal is to have a camera mounted on a linear rail (that moves in the z-direction and is vertical to the plane) that can detect objects on the plane. To my help, I have a laser sensor that constantly measures the height from the plane to the camera, which can be given as an input to the transform.
Any help is appreciated!r.anderssonMon, 11 Nov 2019 03:32:02 -0600http://answers.opencv.org/question/221482/Transform camera positon from one ArUco marker to anotherhttp://answers.opencv.org/question/219606/transform-camera-positon-from-one-aruco-marker-to-another/I'm creating a university project with OpenCV Python and ArUco markers, where I would like to get a (relatively) robust pose estimation for the movement of the camera. I plan on using this for indoor drone flight graphing. For this, I have to transform the camera pose to world coordinates defined by the first seen marker.
I know there must be a transformation matrix between the markers, but I can't seem to figure out, what it is. I am trying with the difference of respective rvecs.
The code for the function in Python:
def TransformBetweenMarkers(tvec_m, tvec_n, rvec_m, rvec_n):
tvec_m = np.transpose(tvec_m) # tvec of 'm' marker
tvec_n = np.transpose(tvec_n) # tvec of 'n' marker
# vector from 'm' to 'n' marker in the camera's coordinate system
dtvec = tvec_m - tvec_n
# get the markers' rotation matrices respectively
R_m = cv2.Rodrigues(rvec_m)[0]
R_n = cv2.Rodrigues(rvec_n)[0]
# camera pose in 'm' marker's coordinate system
tvec_mm = np.matmul(-R_m.T, tvec_m)
# camera pose in 'n' marker's coordinate system
tvec_nn = np.matmul(-R_n.T, tvec_n)
# translational difference between markers in 'm' marker's system,
# basically the origin of 'n'
dtvec_m = np.matmul(-R_m.T, dtvec)
# this gets me the same as tvec_mm,
# but this only works, if 'm' marker is seen
# tvec_nm = dtvec_m + np.matmul(-R_m.T, tvec_n)
# something with the rvec difference must give the transformation(???)
drvec = rvec_m-rvec_n
# transformed to 'm' marker
drvec_m = np.transpose(np.matmul(R_m.T, np.transpose(drvec)))
dR_m = cv2.Rodrigues(drvec_m)[0]
# I want to transform tvec_nn with a single matrix,
# so it would be interpreted in 'm' marker's system
tvec_nm = dtvec_m + np.matmul(dR_m.T, tvec_nn)
# objective: tvec_mm == tvec_nm
This is the best I could get, but there is still an error value of +-0.03 meters between the `tvec_mm` and `tvec_nm` translation values.
Is it possible to get any better with this? Is this even a legit transformation or just a huge coincidence, that it gives approximately the same values? Any ideas?
Thank you!SzepyFri, 11 Oct 2019 13:37:27 -0500http://answers.opencv.org/question/219606/How to get point position in left camera reference with point position in right camera referencehttp://answers.opencv.org/question/214991/how-to-get-point-position-in-left-camera-reference-with-point-position-in-right-camera-reference/Hello everybody,
This might be a simple questions but I am stuck on it, so any help would be really appreciated.
Reading the book "Learning OpenCV" written by Gary Bradski and Adrian Kaehler, I came upon the following relation relating the position of a point in the right camera reference to the left camera reference :
![image description](/upfiles/15619209883731682.png)
Where R and T are the rotation matrix and translation vector between the 2 cameras.
As far as I know, a point in a reference Ox'y'z' can be expressed in the reference Oxyz by computing :
![image description](/upfiles/15619212256187385.png)
![image description](/upfiles/1561921250901772.png)
It is said just afterwards in the book that using
![image description](/upfiles/15619216255254125.png) (which comes from what I just said above),
we can finally get
![image description](/upfiles/15619216707332273.png)
So could anyone explain these relations found in the book?
Thank you.kobekingSun, 30 Jun 2019 14:02:19 -0500http://answers.opencv.org/question/214991/Is there a way to compute a RANSAC based affine transformation?http://answers.opencv.org/question/6747/is-there-a-way-to-compute-a-ransac-based-affine-transformation/I know of `findHomography()`, but that computes all kinds of perspective transformations.
I'd like to restrict the degrees of freedom to translation, rotation and scale.
There is a method `estimateRigidTransform` in the video module, but it's not RANSAC based.
Am I missing something?
Or do I have to implement it on my own?BenThu, 31 Jan 2013 09:56:23 -0600http://answers.opencv.org/question/6747/cv::remap on a pre-rotated imagehttp://answers.opencv.org/question/203726/cvremap-on-a-pre-rotated-image/cv::remap on a pre-rotated image
I am undistorting images from my camera, based on calibration data, using code such as this:
cv::fisheye::estimateNewCameraMatrixForUndistortRectify(cameraMatrix, calibCoeffs, imgSize, cv::Matx33d::eye(), newCamMat, 1);
cv::fisheye::initUndistortRectifyMap(cameraMatrix, calibCoeffs, cv::Matx33d::eye(), newCamMat, imgSize, CV_16SC2, map1, map2);
cv::remap(imgSrc, imgDst, map1, map2, cv::INTER_LINEAR);
Works great. However, I would like to use a hardware feature of my camera to allow image rotation by 90 degrees. For example, if the original image (for which the undistortion works) is 1920x1080, then I would also like to apply the undistortion to rotated frames from the camera (1080x1920).
I (very naively) tried to simply rotate the transformation matrix before using it on the rotated frames, as in:
cv::rotate(map1, map1, cv::ROTATE_90_CLOCKWISE);
cv::rotate(map2, map2, cv::ROTATE_90_CLOCKWISE);
cv::remap(imgSrc, imgDst, map1, map2, cv::INTER_LINEAR);
But this did not provide the expected results.
(Obviously one approach would be to do the rotation after (or along with) the undistortion, but I'd like to avoid that if possible for performance reasons; currently it is being done by the hardware).
Any pointers in the right direction would be appreciated.logidelicTue, 20 Nov 2018 14:09:29 -0600http://answers.opencv.org/question/203726/Convolutional Neural Nets versus kernel-based morphology transformations. What is better?http://answers.opencv.org/question/189203/convolutional-neural-nets-versus-kernel-based-morphology-transformations-what-is-better/If somebody has experience or just tried to compare for himself, can you list the differences? Typical applications, limitations, different output ...
Programmatically, they are the same so I wonder, are there any differences besides the names used?ya_ocv_userThu, 12 Apr 2018 00:52:50 -0500http://answers.opencv.org/question/189203/Using warpPerspective to simulate virtual camera issueshttp://answers.opencv.org/question/189944/using-warpperspective-to-simulate-virtual-camera-issues/Hi guys,
Apologies if this seems trivial - relatively new to openCV.
Essentially, I'm trying to create a function that can take in a camera's image, the known world coordinates of that image, and the world coordinates of some other point 2, and then transform the camera's image to what it would look like if the camera was at point 2. From my understanding, the best way to tackle this is using a homography transformation using the warpPerspective tool.
The experiment is being done inside the Unreal Game simulation engine. Right now, I essentially read the data from the camera, and add a set transformation to the image. However, I seem to be doing something wrong as the image is looking something like this (original image first then distorted image):
**Original Image**
![original image](/upfiles/15244292176869851.png)
**Distorted Image**
![distorted image](/upfiles/15244292432039685.png)
This is the current code I have. Basically, it reads in the texture from Unreal engine, and then gets the individual pixel values and puts them into the openCV Mat. Then I try and apply my warpPerspective transformation. Interestingly, if I just try a simple warpAffine transformation (rotation), it works fine. I would really appreciate any help or guidance any of you may have? Thanks in advance!
ROSCamTextureRenderTargetRes->ReadPixels(ImageData);
cv::Mat image_data_matrix(TexHeight, TexWidth, CV_8UC3);
cv::Mat warp_dst, warp_rotate_dst;
int currCol = 0;
int currRow = 0;
cv::Vec3b* pixel_left = image_data_matrix.ptr<cv::Vec3b>(currRow);
for (auto color : ImageData)
{
pixel_left[currCol][2] = color.R;
pixel_left[currCol][1] = color.G;
pixel_left[currCol][0] = color.B;
currCol++;
if (currCol == TexWidth)
{
currRow++;
currCol = 0;
pixel_left = image_data_matrix.ptr<cv::Vec3b>(currRow);
}
}
warp_dst = cv::Mat(image_data_matrix.rows, image_data_matrix.cols, image_data_matrix.type());
double rotX = (45 - 90)*PI / 180;
double rotY = (90 - 90)*PI / 180;
double rotZ = (90 - 90)*PI / 180;
cv::Mat A1 = (cv::Mat_<float>(4, 3) <<
1, 0, (-1)*TexWidth / 2,
0, 1, (-1)*TexHeight / 2,
0, 0, 0,
0, 0, 1);
// Rotation matrices Rx, Ry, Rz
cv::Mat RX = (cv::Mat_<float>(4, 4) <<
1, 0, 0, 0,
0, cos(rotX), (-1)*sin(rotX), 0,
0, sin(rotX), cos(rotX), 0,
0, 0, 0, 1);
cv::Mat RY = (cv::Mat_<float>(4, 4) <<
cos(rotY), 0, (-1)*sin(rotY), 0,
0, 1, 0, 0,
sin(rotY), 0, cos(rotY), 0,
0, 0, 0, 1);
cv::Mat RZ = (cv::Mat_<float>(4, 4) <<
cos(rotZ), (-1)*sin(rotZ), 0, 0,
sin(rotZ), cos(rotZ), 0, 0,
0, 0, 1, 0,
0, 0, 0, 1);
// R - rotation matrix
cv::Mat R = RX * RY * RZ;
// T - translation matrix
cv::Mat T = (cv::Mat_<float>(4, 4) <<
1, 0, 0, 0,
0, 1, 0, 0,
0, 0, 1, dist,
0, 0, 0, 1);
// K - intrinsic matrix
cv::Mat K = (cv::Mat_<float>(3, 4) <<
12.5, 0, TexHeight / 2, 0,
0, 12.5, TexWidth / 2, 0,
0, 0, 1, 0
);
cv::Mat warp_mat = K * (T * (R * A1));
//warp_mat = cv::getRotationMatrix2D(srcTri[0], 43.0, 1);
//cv::warpAffine(image_data_matrix, warp_dst, warp_mat, warp_dst.size());
cv::warpPerspective(image_data_matrix, warp_dst, warp_mat, image_data_matrix.size(), CV_INTER_CUBIC | CV_WARP_INVERSE_MAP);
cv::imshow("distort", warp_dst);
cv::imshow("imaage", image_data_matrix)
rpSun, 22 Apr 2018 15:41:42 -0500http://answers.opencv.org/question/189944/Row and Col problem when Mat represents 3d point cloudhttp://answers.opencv.org/question/189357/row-and-col-problem-when-mat-represents-3d-point-cloud/The row and col of a Mat to represent 3d point cloud in OpenCV is N * 3, N is the number of the points in the cloud and 3 is the x, y, z coordinate respectively. For example, when I use method loadPLYSimple to load data from PLY file I will get N * 3 Mat, when I use the Flann KDTREE, I need to pass N * 3 Mat as the parameter... But the problem is that when I try to perform a transformation on the data, such as rotation. If we have a 3 * 3 rotation Mat R, and the points Mat pc N * 3, the common way is just R * pc. However, pc is N * 3, so we need to do some extra transpose work. I'm not familiar with OpenCV, I just want to know if there is any better way to do that instead of doing the transpose work each time? Or maybe there is something which I do not understand hidden behind？Thanks.TabFri, 13 Apr 2018 16:28:56 -0500http://answers.opencv.org/question/189357/Recreating lighting based on sample imagehttp://answers.opencv.org/question/184383/recreating-lighting-based-on-sample-image/Hi,
I am wondering if there is any current research/implementation to this problem:
Say given 2 images, one with an object lighted with an (assumingly) uniform lighting, and another image, with the same object, illuminated with a fixed, single light source. Is there a relatively simple way to say, given a 3rd image, with the same object, but from another perspective, to generate a similar lighting?
In order words, is there a way to remodify the 3rd image, such that it looks like how the object would look like under similar lighting conditions as in the 2nd image? Without going through the entire process of building a 3d physics simulation.
Of course, assuming that the objects are purely solid, and ignoring things like transparency and different albedos of the different portion of the object within the images. And that we also have the perspective relations between each individual image.charles1208Sat, 10 Feb 2018 09:20:02 -0600http://answers.opencv.org/question/184383/filtering noise from a transformation matrix, vs a quaternionhttp://answers.opencv.org/question/157463/filtering-noise-from-a-transformation-matrix-vs-a-quaternion/ I am tracking an object, with opencv, in a video stream. The returned values are noisy, so I want to filter them to smooth out the animation curve.
Currently, i have the data as an `Eigen::vector3d` for position, and an `Eigen::Quaterniond` for rotation.
The kalman filtering that I use only takes one value at a time, so i create an instance of the filter for position X, one for position Y, one for position Z.
that bit is easy. My question is:
Am I likely to see any weirdness if I take this same approach with a quaternion? I imagine yes, as the values will change from the filtering, and screw up the rotation.
So, am i better to:
Create a 4x4 transformation matrix from my vector3d and quaternion3d.
filter each of these values.
recreate my quat and vector.
or:
filter the quaternion in a different way.
or:
convert to euler, filter each axis, then convert back.
None of these options seem ideal. But which is most likely to give a decent result?
thank you!
antithingWed, 07 Jun 2017 09:23:43 -0500http://answers.opencv.org/question/157463/How to estimate transformation after hausdorff / shape context matchinghttp://answers.opencv.org/question/156172/how-to-estimate-transformation-after-hausdorff-shape-context-matching/A simular question is asked [here](http://answers.opencv.org/question/66675/how-to-overlay-shapes-in-shape-contexthausdorff-matching/), if i understood the question right:
How can you estimate the location and orientation of a rigid/affine transformed image, after you have extracted the distance and know that the compared images are simular.
I tried estimateRigidTransform, after [casting](https://stackoverflow.com/questions/7386210/convert-opencv-2-vectorpoint2i-to-vectorpoint2f) the vector<points> to vector<point2f> but the resulting Mat keeps empty.
Thank you for your help, the Shape context demo can be found [here](https://github.com/opencv/opencv/blob/master/samples/cpp/shape_example.cpp)
JoeBroeselFri, 02 Jun 2017 09:07:32 -0500http://answers.opencv.org/question/156172/What is the reverse of warpPerspective ?http://answers.opencv.org/question/121228/what-is-the-reverse-of-warpperspective/ Hi all !
At the bottom of this [link](http://docs.opencv.org/3.1.0/da/d6e/tutorial_py_geometric_transformations.html), a transformation extract the sudoku in "full screen".
I would like to draw some text on the extracted sudoku and **re introduce** it in the big picture. Do you know how can I **re introduce** it ?
Here is the code I use:
/**
coordinates is a std::vector<Point2f> with inside:
[topLeft, topRight, bottomLeft, bottomRight]
original is the original picture (the left one in the link) and sudoku is the right one
*/
float w = (float)input.cols;
float h = (float)input.rows;
float hw = w / 2.0f;
float hh = h / 2.0f;
// to points
dst_p[0] = Point2f(0.0f, 0.0f);
dst_p[1] = Point2f(w, 0.0f);
dst_p[2] = Point2f(w, h);
dst_p[3] = Point2f(0.0f, h);
// create perspective transform matrix
Mat trans_mat33 = getPerspectiveTransform(coordinates, dst_p); //CV_64F->double
Mat inv_trans_mat33 = getPerspectiveTransform(dst_p, coordinates);
// perspective transform operation using transform matrix
warpPerspective(input, sudoku, trans_mat33, input.size(), INTER_LINEAR);
sudoku = writeOnPuzzle(sudoku, "012345000000000000000000000000000000000000000000000000000000000000000000000000000");
invert(trans_mat33, inv_trans_mat33);
warpPerspective(sudoku, original, trans_mat33, input.size(), WARP_INVERSE_MAP);
Thank you !BenNGThu, 05 Jan 2017 03:03:11 -0600http://answers.opencv.org/question/121228/why composeRT function is showing error for column vectors?http://answers.opencv.org/question/98867/why-composert-function-is-showing-error-for-column-vectors/ So I'm basically try to get the combination of two rotation and translation transformations.
Mat rvec1(1,3,CV_32F);
rvec1.at<double>(0,0) = 2.3;
rvec1.at<double>(0,1) = 4.5;
rvec1.at<double>(0,2) = 7.8;
Mat rvec2(1,3,CV_32F);
rvec2.at<double>(0,0) = 2.1;
rvec2.at<double>(0,1) = 4;
rvec2.at<double>(0,2) = 1.2;
Mat tvec1(1,3,CV_32F);
tvec1.at<double>(0,0) = 4;
tvec1.at<double>(0,1) = 2;
tvec1.at<double>(0,2) = 3;
Mat tvec2(1,3,CV_32F);
tvec2.at<double>(0,0) = 2;
tvec2.at<double>(0,1) = 3;
tvec2.at<double>(0,2) = 13;
Mat rvec3,tvec3;
//-> Calling composeRT function
composeRT(rvec1, tvec1, rvec2, tvec2, rvec3, tvec3);
But i'm getting this error-
OpenCV Error: Assertion failed (_rvec1->rows == 3 && _rvec1->cols == 1 && CV_ARE_SIZES_EQ(_rvec1, _rvec2)) in cvComposeRT, file /home/kevmepls/opencv-2.4.9/modules/calib3d/src/calibration.cpp, line 407
terminate called after throwing an instance of 'cv::Exception'
what(): /home/kevmepls/opencv-2.4.9/modules/calib3d/src/calibration.cpp:407: error: (-215) _rvec1->rows == 3 && _rvec1->cols == 1 && CV_ARE_SIZES_EQ(_rvec1, _rvec2) in function cvComposeRT
Aborted (core dumped)
If i make the inputs as row vectors, then no errors arises. So why is error coming when inputs are column vectors?kevgeoSat, 23 Jul 2016 02:52:41 -0500http://answers.opencv.org/question/98867/Crop rectangle and apply transformation from image?http://answers.opencv.org/question/98017/crop-rectangle-and-apply-transformation-from-image/Hello all,
i want to detect paper sheet from image.i applied medianBlur, Canny ,dilate,threshold,Etc. algorithms to find.i am able to find sheet but dont know how to crop rectangle and apply transformation
![image description](/upfiles/14678771577240516.png)
this is my code:-
Mat blurred = new Mat();
Imgproc.medianBlur(src, blurred, 9);
// Set up images to use.
Mat gray0 = new Mat(blurred.size(), CvType.CV_8U);
Mat gray = new Mat();
// For Core.mixChannels.
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
List<MatOfPoint2f> rectangles = new ArrayList<MatOfPoint2f>();
List<Mat> sources = new ArrayList<Mat>();
sources.add(blurred);
List<Mat> destinations = new ArrayList<Mat>();
destinations.add(gray0);
// To filter rectangles by their areas.
int srcArea = src.rows() * src.cols();
// Find squares in every color plane of the image.
for (int c = 0; c < 3; c++) {
int[] ch = {c, 0};
MatOfInt fromTo = new MatOfInt(ch);
Core.mixChannels(sources, destinations, fromTo);
// Try several threshold levels.
for (int l = 0; l < N; l++) {
if (l == 0) {
// HACK: Use Canny instead of zero threshold level.
// Canny helps to catch squares with gradient shading.
// NOTE: No kernel size parameters on Java API.
Imgproc.Canny(gray0, gray, 0, CANNY_THRESHOLD);
// Dilate Canny output to remove potential holes between edge segments.
Imgproc.dilate(gray, gray, Mat.ones(new Size(3, 3), 0));
} else {
int threshold = (l + 1) * 255 / N;
Imgproc.threshold(gray0, gray, threshold, 255, Imgproc.THRESH_BINARY);
}
// Find contours and store them all as a list.
Imgproc.findContours(gray, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
int i=0;
for (MatOfPoint contour : contours) {
MatOfPoint2f contourFloat = GeomUtils.toMatOfPointFloat(contour);
double arcLen = Imgproc.arcLength(contourFloat, true) * 0.02;
// Approximate polygonal curves.
MatOfPoint2f approx = new MatOfPoint2f();
Imgproc.approxPolyDP(contourFloat, approx, arcLen, true);
if (isRectangle(approx, srcArea)) {
Imgproc.drawContours(src, contours, i, new Scalar(255, 0, 0), 3);
//rectangles.add(approx);
/*Rect rect = Imgproc.boundingRect(contour);
Log.e("Rectangle Finder:-" + i, "Height:-" + rect.height + ", Width:-" + rect.width + " and Area:-" + rect.area() + "\nX:-" + rect.x + ",Y:-" + rect.y);*/
}
i++;
}
}
i want to select only white papersheet.please help me
Thanks in advanceHardik PatelThu, 07 Jul 2016 02:42:25 -0500http://answers.opencv.org/question/98017/understanding the idea behind some code snippethttp://answers.opencv.org/question/57698/understanding-the-idea-behind-some-code-snippet/regarding some code that I ported from the old opencv C api to the new C++ api, [here](http://answers.opencv.org/question/57179/creation-of-images-with-warpdistortion-due-to-natural-effects-on-camera-lens/?answer=57487#post-id-57487), related to some radial distortion transformation. I am trying to figure out how the following function works.
float calc_shift(float x1,float x2,float cx,float k)
{
float thresh = 1;
float x3 = x1+(x2-x1)*0.5;
float res1 = x1+((x1-cx)*k*((x1-cx)*(x1-cx)));
float res3 = x3+((x3-cx)*k*((x3-cx)*(x3-cx)));
std::cerr<<"x1: "<<x1<<" - "<<res1<<" x3: "<<x3<<" - "<<res3<<std::endl;
if(res1>-thresh and res1 < thresh)
return x1;
if(res3<0){
return calc_shift(x3,x2,cx,k);
}else{
return calc_shift(x1,x3,cx,k);
}
}
the way that the above function is called can be seen below:
int w = src.cols;
int h = src.rows;
xShift = calc_shift(0, Cx - 1, Cx, k);
float newCenterX = w - Cx;
float xShift2 = calc_shift(0, newCenterX - 1, newCenterX, k);
yShift = calc_shift(0, Cy - 1, Cy, k);
float newCenterY = w - Cy;
float yShift2 = calc_shift(0, newCenterY - 1, newCenterY, k);
xScale = (w - xShift - xShift2) / w;
yScale = (h - yShift - yShift2) / h;
where `Cx` and `Cy` correspond to the new coordinates of a given point, usually the center of the image, and `k` some distortion coefficient. I know the outcome, but I want to understand the idea behind regarding the specific values that are used as well.theodoreTue, 17 Mar 2015 21:16:55 -0500http://answers.opencv.org/question/57698/Face stitch - Doubthttp://answers.opencv.org/question/77368/face-stitch-doubt/![image description](/upfiles/1448509739116565.jpg)
Hello guys, good night. I am studying to assemble the mosaic of a face. Does anyone know tell me what function can I use to join the yellow dots on the left of the image with the yellow dots on the right image and assemble the mosaic?I need to modify the image left joins the right. I appreciate the help.
diegomoreiraWed, 25 Nov 2015 21:43:49 -0600http://answers.opencv.org/question/77368/Calculate Log Transformation in RGB Image?http://answers.opencv.org/question/75901/calculate-log-transformation-in-rgb-image/How to calculate log trasformation in RGB image? What is algorithm?m1ndMon, 09 Nov 2015 12:06:09 -0600http://answers.opencv.org/question/75901/perspective transformation with given camera posehttp://answers.opencv.org/question/72020/perspective-transformation-with-given-camera-pose/Hi everyone!
I'm trying to create a program, that I will use to perform some tests.
In this program an 2D image is being displayed in 3D space in the cv:viz window, so user can change camera (viewer) position and orientation.
![image description](/upfiles/1443709792833003.jpg)
After that, program stores camera pose and takes the snaphot of the current view (without coordinates axes):
![image description](/upfiles/14437098062513117.jpg)
An here is the goal:
I have the **snaphot** (perspective view of undetermined plane or part of the plane), **camera pose** (especially its orientation) and **camera parameters**. Using these given values I would like to **perform perspective transformation to compute an ortographic view of this given image** (or its visible part).
I can get the camera object and compute its projection matrix:
camera.computeProjectionMatrix(projectionMatrix);
and then decompose projection matrix:
decomposeProjectionMatrix(subProjMatrix,cameraMatrix, rotMatrix, transVect, rotMatX, rotMatY, rotMatZ);
And what should I do next?
Notice, that I can't use chessboard cornersbecause the image is undetermined (it may be any image) and I can't use the corner points of the image, because user can zoom and translate the camera, so there is posibility, that no image corner point will be visible...
Thanks for any help in advance!pawsThu, 01 Oct 2015 09:41:43 -0500http://answers.opencv.org/question/72020/How do contour "warpPerspective" and "warpAffine"http://answers.opencv.org/question/63028/how-do-contour-warpperspective-and-warpaffine/findContours(imgBinary, allContours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
I haved the allContours, and Matrix of Perspective and Affine Transformation
How do contours Perspective(2D) and Affine Transformation???
Which function use???
Use direct "perspectiveTransform", have error
vector<vector<Point> >contours1;
perspectiveTransform(allContours, contours1, H);
nistarMon, 01 Jun 2015 11:00:16 -0500http://answers.opencv.org/question/63028/2 Video streams from 2 different cameras->Transformation between views?http://answers.opencv.org/question/62354/2-video-streams-from-2-different-cameras-transformation-between-views/Hi guys!
Let's say I have 2 cameras.They are both looking at a scene where a marker exists,however these images are not the same(there is an offset between these 2cameras,e.g one is on top of the other).
Now let's say I render a virtual cube on top of the marker that is tracked by one of the cameras.Next I want to draw the same cube,in the second camera's view.However for it to work I need to apply some transformations to the original cube.Can you tell me how can I find these transformations via openCV?
What steps should I do?
**Update 1:**
Both cameras are mounted on an oculus and they move in the same way.
marios.bThu, 21 May 2015 06:11:17 -0500http://answers.opencv.org/question/62354/Track with monocular camera-See with stereo camerashttp://answers.opencv.org/question/61011/track-with-monocular-camera-see-with-stereo-cameras/Hi guys!
I want to build an application, so that the user can wear Oculus+Stereo cameras in front of oculus+RGB-D camera mounted also on top of oculus. The goal will be to use RGB-D camera to track a marker in a scene and then I want the user to see the virtual content through the stereo cameras instead of the RGB-D Camera.
So RGB-D camera will do all the tracking stuff,but user will see virtual objects through stereo cameras.
What exactly should I do to achieve this?Do I need to transform the virtual objects from RGB-D camera to the stereo cameras?How can I do it?Any resources/tips or similar projects? marios13Fri, 01 May 2015 09:07:40 -0500http://answers.opencv.org/question/61011/tformarray matlab function equivalent in opencvhttp://answers.opencv.org/question/57591/tformarray-matlab-function-equivalent-in-opencv/ tmap_B = cat(3,u,v);
resamp = makeresampler('linear','fill');
J2 = tformarray(I,[],resamp,[2 1],[1 2],[],tmap_B,.3);
I want to port the above matlab code to corresponding opencv, where u and v are two 2D matrices. The [cat()](http://se.mathworks.com/help/matlab/ref/cat.html) function corresponds to the concatenate procedure, from a short research I came up with the `hconcat()` and
`vconcat()` but these are for horizontal and vertical concatenation respectively. Is [merge()](http://docs.opencv.org/modules/core/doc/operations_on_arrays.html#merge) what I should use instead or there is something else?
As for the [tformarray()](http://se.mathworks.com/help/images/ref/tformarray.html) corresponds to a spatial transformation to an array A in order to produce an array B. I am trying to figure out which is the equivalent from the opencv transformation functions. Any idea?theodoreMon, 16 Mar 2015 19:08:29 -0500http://answers.opencv.org/question/57591/Image Transformation OpenCV/C++http://answers.opencv.org/question/55985/image-transformation-opencvc/ Hello,
I'm trying to perform a transformation, or perspective correction, on the image with the chess board features as follows:
![image description](/upfiles/14247404473169105.png)
However, already tried in several places, including in the book Learning OpenCV and have not found a way to do this. If anyone can help me with this I thank you!jefferson-sepMon, 23 Feb 2015 19:17:35 -0600http://answers.opencv.org/question/55985/FindHomography usagehttp://answers.opencv.org/question/55605/findhomography-usage/Hi all,
i'm completely new, so i think it's very simple my problem.
**Context**
I'm using OpenCV library for Android, i'm capturing a video with my device (fixed in a position, not in movement), with a component of my program i can determine 4 point from the scene. So at the end I have a 640x480 int array , and i have 4 values (indexes) of the four point in the scene.
**Problem**
I want to do this kind of transformation:
![transformation](http://qvision.sourceforge.net/rectification_homography.png)
**Question**
What do i have to do to make such a ("plane") transformation, in order to watch the scene from above? What i'm trying to saying is that: given 4 indexes of a rgb array that identify 4 real points of the scene, what do i have to pass to findHomography function? And after that, what do i have to do with Mat object to apply the change in real time?
I actually need detailed steps, because I'm completely new.
Thanks a lot,.
Sorry i'm new to Computer Vision. psykomantisitaWed, 18 Feb 2015 12:42:38 -0600http://answers.opencv.org/question/55605/How could I use a line as a structuring element for morphology transformations?http://answers.opencv.org/question/51117/how-could-i-use-a-line-as-a-structuring-element-for-morphology-transformations/I'm using the documentation example [*More Morphology Transformations*](http://docs.opencv.org/doc/tutorials/imgproc/opening_closing_hats/opening_closing_hats.html#morphology-2) (newbie here). In the *kernel type*, you can select between cross/ellipse/rectangle. I would like to use a 5x1 **line** or something like that. I found this [answer](http://answers.opencv.org/question/5448/rotated-ellipse-as-structuring-element/?answer=5449#post-id-5449) which discusses a rotated ellipse as a *structuring element* and this other answer about using a external file as a structuring element. Both of which I want to try.
I'm looking at [getStructuringElement](http://docs.opencv.org/modules/imgproc/doc/filtering.html?highlight=getstructuringelement#getstructuringelement); I'm afraid this is beyond me at the moment. I can't connect the dots here as in integrate the answers to the example so that I could specify a line to be used as kernel either from the code or by supplying an image file. I reproduce the original example:
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <stdlib.h>
#include <stdio.h>
using namespace cv;
/// Global variables
Mat src, dst;
int morph_elem = 0;
int morph_size = 0;
int morph_operator = 0;
int const max_operator = 4;
int const max_elem = 2;
int const max_kernel_size = 21;
char* window_name = "Morphology Transformations Demo";
/** Function Headers */
void Morphology_Operations( int, void* );
/** @function main */
int main( int argc, char** argv )
{
/// Load an image
src = imread( argv[1] );
if( !src.data )
{ return -1; }
/// Create window
namedWindow( window_name, CV_WINDOW_AUTOSIZE );
/// Create Trackbar to select Morphology operation
createTrackbar("Operator:\n 0: Opening - 1: Closing \n 2: Gradient - 3: Top Hat \n 4: Black Hat", window_name, &morph_operator, max_operator, Morphology_Operations );
/// Create Trackbar to select kernel type
createTrackbar( "Element:\n 0: Rect - 1: Cross - 2: Ellipse", window_name,
&morph_elem, max_elem,
Morphology_Operations );
/// Create Trackbar to choose kernel size
createTrackbar( "Kernel size:\n 2n +1", window_name,
&morph_size, max_kernel_size,
Morphology_Operations );
/// Default start
Morphology_Operations( 0, 0 );
waitKey(0);
return 0;
}
/**
* @function Morphology_Operations
*/
void Morphology_Operations( int, void* )
{
// Since MORPH_X : 2,3,4,5 and 6
int operation = morph_operator + 2;
Mat element = getStructuringElement( morph_elem, Size( 2*morph_size + 1, 2*morph_size+1 ), Point( morph_size, morph_size ) );
/// Apply the specified morphology operation
morphologyEx( src, dst, operation, element );
imshow( window_name, dst );
}
oCV_newbieTue, 18 Nov 2014 22:58:56 -0600http://answers.opencv.org/question/51117/