OpenCV Q&A Forum - RSS feedhttp://answers.opencv.org/questions/OpenCV answersenCopyright <a href="http://www.opencv.org">OpenCV foundation</a>, 2012-2018.Sat, 18 Nov 2017 14:40:28 -0600Convert pixel position to world direction?http://answers.opencv.org/question/66047/convert-pixel-position-to-world-direction/I have calibrated my camera with a checkerboard and achieved the distortion parameters and Intrinsic matrix of my camera.
Using these I have estimated the camera position and orientation using solvePnP against a known known set of reference points.
Now I want to find the world direction, from the camera toward a blob I am detecting inside my image. So I want to convert a pixel position to a 3D vector in that direction.
I want the direction so I can determine where in the world a ball is located. I have two cameras. If both of them can see the blob I will find the point nearest both lines and if only one can observe the ball I will just use the directions crossing with a the ground plane.
I am using blob detection in HSV colorspace to find the ball.
Any ideas on how to continue?
Kind regards
Jesper
TAXfromDKSat, 11 Jul 2015 10:25:21 -0500http://answers.opencv.org/question/66047/OpenGL 4x4 camera matrix, OpenCV ?x? camera matrixhttp://answers.opencv.org/question/178674/opengl-4x4-camera-matrix-opencv-x-camera-matrix/In OpenGL, the camera matrix is a 4x4 matrix. Is the camera matrix in OpenCV a 4x4 matrix as well?
The following is the code needed to make a very simple camera matrix. Compatible with OpenGL ES 2.0 and higher. See the camera matrix applied in the full code https://github.com/sjhalayka/blind_poker:
float projection_modelview_mat[16];
init_perspective_camera(y_fov_degrees,
static_cast<float>(screen_width)/static_cast<float>(screen_height),
0.01f, 2.0f, // Z near, far distances.
0, 0, 1, // Camera position.
0, 0, 0, // Look at position.
0, 1, 0, // Up direction vector.
projection_modelview_mat);
... where ...
void get_perspective_matrix(float fovy, float aspect, float znear, float zfar, float (&mat)[16])
{
// https://www.opengl.org/sdk/docs/man2/xhtml/gluPerspective.xml
const float pi = 4.0f*atanf(1.0);
// Convert fovy to radians, then divide by 2
float f = 1.0f / tan(fovy/360.0*pi);
mat[0] = f/aspect; mat[4] = 0; mat[8] = 0; mat[12] = 0;
mat[1] = 0; mat[5] = f; mat[9] = 0; mat[13] = 0;
mat[2] = 0; mat[6] = 0; mat[10] = (zfar + znear)/(znear - zfar); mat[14] = (2.0f*zfar*znear)/(znear - zfar);
mat[3] = 0; mat[7] = 0; mat[11] = -1; mat[15] = 0;
}
void get_look_at_matrix(float eyex, float eyey, float eyez,
float centrex, float centrey, float centrez,
float upx, float upy, float upz,
float (&mat)[16])
{
// https://www.opengl.org/sdk/docs/man2/xhtml/gluLookAt.xml
vertex_3 f, up, s, u;
f.x = centrex - eyex;
f.y = centrey - eyey;
f.z = centrez - eyez;
f.normalize();
up.x = upx;
up.y = upy;
up.z = upz;
up.normalize();
s = f.cross(up);
s.normalize();
u = s.cross(f);
u.normalize();
mat[0] = s.x; mat[4] = s.y; mat[8] = s.z; mat[12] = 0;
mat[1] = u.x; mat[5] = u.y; mat[9] = u.z; mat[13] = 0;
mat[2] = -f.x; mat[6] = -f.y; mat[10] = -f.z; mat[14] = 0;
mat[3] = 0; mat[7] = 0; mat[11] = 0; mat[15] = 1;
float translate[16];
translate[0] = 1; translate[4] = 0; translate[8] = 0; translate[12] = -eyex;
translate[1] = 0; translate[5] = 1; translate[9] = 0; translate[13] = -eyey;
translate[2] = 0; translate[6] = 0; translate[10] = 1; translate[14] = -eyez;
translate[3] = 0; translate[7] = 0; translate[11] = 0; translate[15] = 1;
float temp[16];
multiply_4x4_matrices(mat, translate, temp);
for(size_t i = 0; i < 16; i++)
mat[i] = temp[i];
}
void multiply_4x4_matrices(float (&in_a)[16], float (&in_b)[16], float (&out)[16])
{
/*
matrix layout:
[0 4 8 12]
[1 5 9 13]
[2 6 10 14]
[3 7 11 15]
*/
out[0] = in_a[0] * in_b[0] + in_a[4] * in_b[1] + in_a[8] * in_b[2] + in_a[12] * in_b[3];
out[1] = in_a[1] * in_b[0] + in_a[5] * in_b[1] + in_a[9] * in_b[2] + in_a[13] * in_b[3];
out[2] = in_a[2] * in_b[0] + in_a[6] * in_b[1] + in_a[10] * in_b[2] + in_a[14] * in_b[3];
out[3] = in_a[3] * in_b[0] + in_a[7] * in_b[1] + in_a[11] * in_b[2] + in_a[15] * in_b[3];
out[4] = in_a[0] * in_b[4] + in_a[4] * in_b[5] + in_a[8] * in_b[6] + in_a[12] * in_b[7];
out[5] = in_a[1] * in_b[4] + in_a[5] * in_b[5] + in_a[9] * in_b[6] + in_a[13] * in_b[7];
out[6] = in_a[2] * in_b[4] + in_a[6] * in_b[5] + in_a[10] * in_b[6] + in_a[14] * in_b[7];
out[7] = in_a[3] * in_b[4] + in_a[7] * in_b[5] + in_a[11] * in_b[6] + in_a[15] * in_b[7];
out[8] = in_a[0] * in_b[8] + in_a[4] * in_b[9] + in_a[8] * in_b[10] + in_a[12] * in_b[11];
out[9] = in_a[1] * in_b[8] + in_a[5] * in_b[9] + in_a[9] * in_b[10] + in_a[13] * in_b[11];
out[10] = in_a[2] * in_b[8] + in_a[6] * in_b[9] + in_a[10] * in_b[10] + in_a[14] * in_b[11];
out[11] = in_a[3] * in_b[8] + in_a[7] * in_b[9] + in_a[11] * in_b[10] + in_a[15] * in_b[11];
out[12] = in_a[0] * in_b[12] + in_a[4] * in_b[13] + in_a[8] * in_b[14] + in_a[12] * in_b[15];
out[13] = in_a[1] * in_b[12] + in_a[5] * in_b[13] + in_a[9] * in_b[14] + in_a[13] * in_b[15];
out[14] = in_a[2] * in_b[12] + in_a[6] * in_b[13] + in_a[10] * in_b[14] + in_a[14] * in_b[15];
out[15] = in_a[3] * in_b[12] + in_a[7] * in_b[13] + in_a[11] * in_b[14] + in_a[15] * in_b[15];
}
/*
void multiply_4x4_matrices(float (&in_a)[16], float (&in_b)[16], float (&out)[16])
{
for(int i = 0; i < 4; i++)
{
for(int j = 0; j < 4; j++)
{
out[4*i + j] = 0;
for (int k = 0; k < 4; k++)
out[4*i + j] += in_a[4*k + j] * in_b[4*i + k];
}
}
}
*/
void init_perspective_camera(float fovy, float aspect, float znear, float zfar,
float eyex, float eyey, float eyez, float centrex, float centrey,
float centrez, float upx, float upy, float upz,
float (&projection_modelview_mat)[16])
{
float projection_mat[16];
get_perspective_matrix(fovy, aspect, znear, zfar, projection_mat);
float modelview_mat[16];
get_look_at_matrix(eyex, eyey, eyez, // Eye position.
centrex, centrey, centrez, // Look at position (not direction).
upx, upy, upz, // Up direction vector.
modelview_mat);
multiply_4x4_matrices(projection_mat, modelview_mat, projection_modelview_mat);
}
... and the vertex and fragment shaders are:
// vertex shader
attribute vec3 position;
attribute vec2 tex_coord;
uniform mat4 mvp_matrix;
varying vec2 frag_tex_coord;
void main()
{
frag_tex_coord = tex_coord;
gl_Position = mvp_matrix*vec4(position, 1);
}
// fragment shader
uniform sampler2D tex;
varying mediump vec2 frag_tex_coord;
void main()
{
gl_FragColor = texture2D(tex, frag_tex_coord);
}sjhalaykaSat, 18 Nov 2017 14:40:28 -0600http://answers.opencv.org/question/178674/Camera Matrixhttp://answers.opencv.org/question/122818/camera-matrix/ Are the fx,fy,cx and cy for a camera constant? or do they differ with different pictures when clicked?PrototypeMon, 16 Jan 2017 02:06:31 -0600http://answers.opencv.org/question/122818/Calculation 4x4 matrix of camerahttp://answers.opencv.org/question/101909/calculation-4x4-matrix-of-camera/Hi everyone!
I have World Frame. I have diodes and I know coordinates of them (X,Y,Z) in this World Frame. I have camera, which takes pictures. I need to calculate matrix 4x4 of camera in World Frame.
Which function of OpenCV should I use? How many diodes do I need?
I need full matrix 4x4 (X,Y,Z + RotMatrix 3x3).Vlad222Thu, 08 Sep 2016 10:12:48 -0500http://answers.opencv.org/question/101909/Understanding the camera matrixhttp://answers.opencv.org/question/89786/understanding-the-camera-matrix/ Hello all,
I used a chessboard calibration procedure to obtain a camera matrix using OpenCV and python based on this tutorial: http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_calib3d/py_calibration/py_calibration.html
I ran through the sample code on that page and was able to reproduce their results with the chessboard pictures in the OpenCV folder to get a camera matrix.
I then tried the same procedure with my own checkerboard grid and camera, and I obtained the following matrix:
mtx = [1535 0 638
0 1536 204
0 0 1]
I am trying to better understand these results, based on the camera sensor and lens I am using.
Based on: http://ksimek.github.io/2013/08/13/intrinsic/
Fx = fx * W/w
Fx = focal length in mm
W = sensor width in mm
w = image width in pixels
fx = focal length in pixels
The size of my images: 1264 x 512 (width x height)
I am using the following lens: http://www.edmundoptics.com/imaging/imaging-lenses/edmund-optics-designed-lenses/megapixel-finite-conjugate-video-imaging-lenses/58203/
This has focal length 8 mm.
I am using a FL3-U3-13Y3 camera from PtGrey (https://www.ptgrey.com/flea3-13-mp-mono-usb3-vision-vita-1300-camera), which has an image width of 12 mm, according to this picture:
![image description](/upfiles/1457570354722283.png)
From the camera matrix, fx is the element in the first row, first column. So above, fx = 1535. In short:
fx = 1535 pixels (from camera matrix I obtained)
w = 1264 pixels (image size I set)
W = 12 mm (from datasheet)
Fx = 8 mm (from datasheet)
Using: Fx = fx * W/w, we would expect
Fx = 1535 * 12 / 1264 = 14.57 mm
But the actual lens is 8 mm. Why the discrepancy?
I would think that the actual size of a chess grid would have to be known, but I did not see mention of manipulation of that in the tutorial link I provided. I basically had to scale down the chessboard grid so that it would work with my camera setup.
I would appreciate any help or insight on this.
Thanks in advance
**EDIT:**
Actually to be more specific, the lens has a maximum camera sensor format of 1/3", while the camera sensor format is 1/2". I found an article on this: http://www.cambridgeincolour.com/tutorials/digital-camera-sensor-size.htm
Focal length multiplier = (1/2) / (1/3) = 1.5
Focal length of lens as listed on datasheet = 8 mm
Equivalent focal length of lens= 1.5 * 8 mm = 12 mm
Still, 12 mm is off from 14.57 mm. Am I not factoring something else in my calculation? Could this be happening from bad images that still happen to find the chessboard corners?
Below is an example image:
![image description](/upfiles/146194871634683.png)solarflareWed, 09 Mar 2016 18:45:05 -0600http://answers.opencv.org/question/89786/Camera Matrix with Extra Optical Elementshttp://answers.opencv.org/question/93798/camera-matrix-with-extra-optical-elements/ Hello,
I am using a camera sensor with a fixed focal length lens that has a distortion on it. Normally I would use the OpenCV chessboard routine as is to correct for it, but in this case I have extra optical elements. The camera goes through a mirror and then a reflection prism that splits the image. Each split image then goes through its own mirror. Essentially this results in three regions: a left region of interest that corresponds to one of the split images, a right region of interest that corresponds to the other split image, and some dead space in between that I can't detect. My question is how should the camera matrix be obtained?
1. With the camera alone without any optical elements aside from the lens. Perform the chessboard test, obtain the camera matrix, and use this matrix after attaching the camera back to the rest of the system.
2. Obtain a separate camera matrix for the left and right region of interest, after the optical elements. I believe theoretically the left and camera matrix should be the same, but there may be slight differences due to alignment issues of the optics and/or camera. But is it valid to do a chessboard test to obtain a camera matrix if center of the image does not correspond with the center of the camera?
3. Some other method.
I don't think the reflection mirrors should be causing too much optical aberration.
Thanks in advance.
**::::::::::::::::::EDIT:::::::::::::::::**
----------------------------------------
So as a point of comparison, I did a test using the first two approaches I mentioned above and here are the resulting intrinsic matrices I obtained from the data.
----------
**Calibration using only camera without optical elements:**
mtx = [[ 1.53451091e+03 0.00000000e+00 6.37547946e+02]
[ 0.00000000e+00 1.53575661e+03 2.03955413e+02]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
dist = [[ -2.90082167e-01 -1.25493796e+00 -6.68277712e-04 5.48729228e-03 3.24470291e+00]]
----------
**Left region calibration with optical elements:**
mtx = [[ 1.63242750e+03 0.00000000e+00 7.04154505e+02]
[ 0.00000000e+00 1.62893516e+03 2.66533979e+02]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
dist = [[ -5.39250573e-01 6.71474407e+00 -7.75939341e-03 3.58186934e-03 -5.96964357e+01]
----------
**Right region calibration with optical elements:**
mtx = [[ 1.56749908e+03 0.00000000e+00 3.84091853e+02]
[ 0.00000000e+00 1.55558685e+03 1.83966722e+02]
[ 0.00000000e+00 0.00000000e+00 1.00000000e+00]]
dist = [[ -5.01140894e-01 9.92103465e+00 -1.17531442e-02 -1.02322088e-02 -1.86869431e+02]]
So some key points:
1. There are some differences in the focal lengths measured in each case. Probably has to do with the alignment of the optical elements.
2. The rightmost column of the camera matrix is different in each case. This makes sense, because when the camera is calibrated on its own, it has a whole image to work with. But when the camera goes through the splitting optics, in each case there is only half an image. Though since the image is split across the middle, I would have expected the y-coordinate (2nd row, 3rd column) to be the same for the left and right camera matrices.
3. The distortion coefficients are different. Though I think it's hard to compare in the different cases. Could simply be that the different regions have different distortion factors?solarflareTue, 26 Apr 2016 09:25:38 -0500http://answers.opencv.org/question/93798/Camera calibration for distortion correctionhttp://answers.opencv.org/question/74305/camera-calibration-for-distortion-correction/ Hello everybody,
First, I figured out some problems in understanding the distortion correction in openCV.
In different sources one can find different formulas to discribe the distortion:
http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html
http://docs.opencv.org/master/dc/dbb/tutorial_py_calibration.html#gsc.tab=0
The first 4 equations in both tutorials are different from each other, as one describes the corrected pixel coordinates and the other one describes the distorted ones. This discrepancy can also be found in different books, theses and dissertations. Can anyone explain where it comes from?
Second, I am wondering what are typical values for the distortion vector in just slightly distortet pictures? The distortion in my picture is not visible - it matters for measurements anyways. Nevertheless the calculated distortion vector elements vary between 10^-2 and 10^+4! I am wondering how theses values can be that big. Any ideas?
Furthermore, there is another issue for me in understanding the camera matrix as reportet in both links:
f_x and f_y are almost identical in my case, but far far away from the real focal length in mm. Is there any scalining factor to compare both in order to validate the matrix result? Passing the matrix to calibrationMatrixValues() gives me a focal length about 3 times bigger than the actual focal length of my camera.
I'd be really greatful for answers on any of the issues above :)
Best regards
DanielDaniel_1985Mon, 26 Oct 2015 05:47:52 -0500http://answers.opencv.org/question/74305/perspective transformation with given camera posehttp://answers.opencv.org/question/72020/perspective-transformation-with-given-camera-pose/Hi everyone!
I'm trying to create a program, that I will use to perform some tests.
In this program an 2D image is being displayed in 3D space in the cv:viz window, so user can change camera (viewer) position and orientation.
![image description](/upfiles/1443709792833003.jpg)
After that, program stores camera pose and takes the snaphot of the current view (without coordinates axes):
![image description](/upfiles/14437098062513117.jpg)
An here is the goal:
I have the **snaphot** (perspective view of undetermined plane or part of the plane), **camera pose** (especially its orientation) and **camera parameters**. Using these given values I would like to **perform perspective transformation to compute an ortographic view of this given image** (or its visible part).
I can get the camera object and compute its projection matrix:
camera.computeProjectionMatrix(projectionMatrix);
and then decompose projection matrix:
decomposeProjectionMatrix(subProjMatrix,cameraMatrix, rotMatrix, transVect, rotMatX, rotMatY, rotMatZ);
And what should I do next?
Notice, that I can't use chessboard cornersbecause the image is undetermined (it may be any image) and I can't use the corner points of the image, because user can zoom and translate the camera, so there is posibility, that no image corner point will be visible...
Thanks for any help in advance!pawsThu, 01 Oct 2015 09:41:43 -0500http://answers.opencv.org/question/72020/The coordinate system of pinhole camera modelhttp://answers.opencv.org/question/31470/the-coordinate-system-of-pinhole-camera-model/Recently, I have been studying the pinhole camera model for several days but I was confused with the model provided by OpenCV and "Multiple View geometry in computer vision" which is a famous textbook.
I know that the following photo is a simplified model which switches the position of the image plane and the camera frame. Basically,for better illustration and understanding and Taking consideration of the principal point (u0,v0), the relation between two frames is
x=f(X/Z)+u0 and
y=f(Y/Z)+vo.
![image description](/upfiles/1397050375379081.png)
However,I was really confused because normally the image coordinate is in the form of the 4th quadrant coordinate as the following one!
Could I directly substitute the (x,y) in the following definition to the above "equivalent" pinhole model which is not really persuasive?
![image description](/upfiles/13970504447802913.gif)
Besides, If an object is in the region (+X,+Y) quadrant in the camera coordinate (of course, Z>f), in the equivalent model, it should appear on the right-half plane of the image coordinate. However, such object in the image taken by a normal camera, it is supposed to be located on the left-half. Therefore, for me this model is not reasonable.
Finally, I tried to derive based on the original model as the following one.
![image description](/upfiles/13970504813232063.png)
The result is
x1=-f(X/Z) and
y1=-f(Y/Z). Then, I tried to find the relation between (x2,y2)-coordinate and the camera coordinate. The result is
x2=-f(X/Z)+u0 and
y2=-f(Y/Z)+vo.
Between (x3,y3)-coordinate and the camera coordinate, the result is
x3=-f(X/Z)+u0 and
y3=f(Y/Z)+vo.
no matter which coordinate system i tried, none of them is in the form of
x=f(X/Z)+u0 and
y=f(Y/Z)+vo, which are provided by some CV textbooks.
Besides, the projection results on (x2,y2)-coordinate or (x3,y3)-coordinate are also not reasonable because of the same reason- an object in the (+X,+Y,+Z) region in the camera coordinate should "appear" on the left-half plane of the image taken by a camera.
Could anyone indicate what I was misunderstood with and I will try to derive several times more and post the answer when someone else help me figure this issue out.
Thank you in advance!!
AlexAlexofNTUWed, 09 Apr 2014 08:37:31 -0500http://answers.opencv.org/question/31470/NEWBIE calibrateCamera and calibrationMatrixValueshttp://answers.opencv.org/question/17590/newbie-calibratecamera-and-calibrationmatrixvalues/hi- i am a total newb to opencv- and to python as well. i'm trying to get a better understanding of the workings of the calibrateCamera and calibrationMatrixValues functions. i'm stepping through someone else's (working) python code. thanks in advance for your patience.
i am ultimately trying to use calibrationMatrixValues to extract the fovx and fovy from a previously (successfully) solved camera calibration using calibrateCamera. these are the settings used (where geoPoints, imgPoints, pWidth, and pHeight have been determined).
calibrateCamera((geoPoints,imgPoints),width=pWidth, height=pHeight, useIntrinsicGuess=True, fixAspectRatio=True, zeroTangentDist=True, fixPrincipalPoint=False)
calibrateCamera - the output camera matrix i get is indeed a 3x3 matrix, but it is a rotation matrix that is being used to calculate my camera's extrinsics. i think i am looking for output that is a projection matrix- so i can feed that into calibrationMatrixValues.
calibrationMatrixValues - where can i find the camera matrix to input? imageSize, is this an array of two values [x,y]? i know the image size, i'm not sure how to format it properly.
and finally- apertureWidth and apertureHeight. these i am stuck on. any guess as to how i would extract or calculate these values from calibrateCamera?
---
in stepping through the source code for the calibrationMatrixValues function i find:
fovx = 2 * atan(imgWidth / (2 * alphax)) * 180.0 / CV_PI;
fovy = 2 * atan(imgHeight / (2 * alphay)) * 180.0 / CV_PI;
alphax and alphay are pulled directly from the input camera matrix. so, a big problem for me is the matrix i feed it looks like a rotation matrix, and this produces very unusable results.
thanks very muchgardyllFri, 26 Jul 2013 10:10:25 -0500http://answers.opencv.org/question/17590/Can we find Camera matrix, without calibration?http://answers.opencv.org/question/14731/can-we-find-camera-matrix-without-calibration/Hi,
I would like to know if it is possible to know the camera calibration matrix anyhow , just by knowing it's specifications , without using camera calibration???newbie111Thu, 06 Jun 2013 01:27:52 -0500http://answers.opencv.org/question/14731/Results of camera calibration varyhttp://answers.opencv.org/question/7554/results-of-camera-calibration-vary/Hi,
I have a problem during camera calibration using chess board. I'm using C++ calibrateCamera method to find out the intrinsic matrix. The problem is the fx, fy are varying from 700 to 1100 depending on the distance from camera the cx, cy vary as well. My question is, can it be caused by poor camera quality(using some no-name webcam with resolution 640x480). **Can this poor resolution be the only problem**??
I need to have really accurate intrinsic matrix and in this case it seems I will need to calculate it by myself:(
Just to be sure for webcam with fixed focus length, the intrinsic matrix should be always the same isn't right???
Thanks for clarification.
MartinMartinSun, 17 Feb 2013 14:18:26 -0600http://answers.opencv.org/question/7554/