Coordinates and 3D reconstruction after stereoCalibration

I am new to OpenCV stereovision and several questions after running the sample stereo_calib.cpp using the sample leftImage and rightImage pairs:

1. I have tried to reconstruct 3D points using the image pairs for calibration: imagePoints[0] and imagePoints[1]. We should be able to get a point cloud in the 3D object space consisting of corners in all board orientations. what's the procedure to do this?
2. Some documents said we need to rectify the two images first, then find the matching feature like corners. Because we already have matched image pairs (distorted) that are used for calibration, I do not want to run matching again and just want to use the image pairs for 3D reconstruction. I also try to avoid remapping the entire images to save image processing time. to do this, first thing is to understand what in rmap matrix. After "initUndistortRectifyMap(cameraMatrix[0], distCoeffs[0], R1, P1, imageSize, CV_16SC2, rmap[0][0], rmap[0][1])" at line 278 and 279, rmap[0][0] has 3.15 MB while rmap[0][1] has 1.61 MB. From openCV document, one mapping is for Y and one map for X, then they should be the same size. Why do they have different size? how to use them without using remap() function?
3. Where is the origin of the 3D object coordinate system located after stereo calibration? Thanks, Jim
edit retag close merge delete

Sort by » oldest newest most voted

Thank you, LBerger. I used triangulatePoints() to reconstruct 3D points successfully using undistorted image point pairs, although accuracy is not satisfactory yet.
The origin of the world coordinated system seems to be the principal point (lens center) of the left camera. Still do not know how to use map1 and map2. Th document said: "map1 – The first map of either (x,y) points or just x values having the type CV_16SC2 , CV_32FC1 , or CV_32FC2 . See convertMaps() for details on converting a floating point representation to fixed-point for speed. map2 – The second map of y values having the type CV_16UC1 , CV_32FC1 , or none (empty map if map1 is (x,y) points), respectively." Most sample codes I found in www is to use map1 fro x replacement and map2 for Y-replacement. It is obvious that although Map1 has x and y, but map2 is not empty. Do not know how they are used in remap(). Is there any source code in Remap() or demonstrate how to use map1 and map2?

more

For the remap,instead use CV_32FC1 in your initUndistortRectifyMap, which should give you a set of images X and Y that you use the way you would expect. Passing CV_16SC2 is a compressed, lower precision version that uses less space, but is accurate enough for image processing work.

( 2017-03-24 18:39:57 -0600 )edit

Thanks. As described in the documents, there are two cases for remap() function: in the first case, if Map 1 is CV_32FC1 or CV_16FC2, Map1 is double the size of source matrix and Map2 should be empty; In the second case, both Map1 and Map2 are CV_16C1. But in Stereo_calib.cpp, Map1 is in CV_32FC1, same size as the source image. If double sized Map1 is enough for remap() function, why map2 is not empty? In this case, how to use both Map1 and Map2?

( 2017-03-29 12:48:43 -0600 )edit

CV_32FC1 and CV_32FC1 are literally x and y coordinates. To find the value that goes in (0,0), you read the value at src(map1(0,0), map2(0,0)).

CV_32FC2 and Nothing is just those two maps as channels of one Mat object.

CV_16UC2 and CV_16UC1 is a lower-precision version of the CV_32FC2. The CV_16UC1 map stores interpolation values. This is not really human readable, but it uses less memory.

( 2017-03-29 18:26:27 -0600 )edit

About 1 you can use triangulate method (you need to undistort points)

About initUndistortRectifyMap first map is 2d map and second a 1D map. First is x,y mapping and I think second map is an interpolation map

you can find explanation in 2.4 docor 3.2 doc

About coordinate system see StereoCalibrate :

R   Output rotation matrix between the 1st and the 2nd camera coordinate systems.


Coordinate system is described in Detailed Description of calib3d module (sorry there is no link to detail description available)

You can try example in opencv stereomatch and stereocalib and stereobm

About remap I wrote an example to zoom an image. Source image is 3x3 matrix and destination a 6x6 matrix. First map is CV_32FC2 which is converted in 16SC2 and 16SC1 map.

#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main(int argc, char **argv)
{
Mat x=(Mat_<uchar>(3,3)<<10,20,30,60,60,60,40,50,60);
Mat map1(2*x.rows,2*x.cols,CV_32FC2);
Mat map1a;
Mat map2a;

for (int i=0;i<map1.rows;i++)
for (int j = 0; j<map1.cols; j++)
{
Vec2f v(j/2.,i/2.);
map1.at<Vec2f>(i,j)=v;
}
cout <<"Original image \n"<< x << "\n";
cout<< "Map CV_32FC2 for a zoom  X 2 \nResults [x0,y0,x1,y0,...]\n";
cout<<map1<<"\n";
Mat y;
remap(x,y,map1,Mat(), INTER_LINEAR);
cout<< "Image zoom x2 using CV_32FC2\n";
cout << y << "\n";
// Convert a 32FC2 map in 16SC2+Interpolation map
convertMaps(map1,Mat(),map1a,map2a,CV_16SC2);
cout <<" Map CV_16SC2 for a zoom  X 2 \nResults[x0, y0, x1, y0, ...]\n"<< map1a << "\n";
cout <<" Interpolation tab \n"<< map2a << "\n";

remap(x, y, map1a, map2a, INTER_LINEAR);
cout << y << "\n";
return 0;
}


results are

Original image
[ 10,  20,  30;
60,  60,  60;
40,  50,  60]
Map CV_32FC2 for a zoom  X 2
Results [x0,y0,x1,y0,...]
[0, 0, 0.5, 0, 1, 0, 1.5, 0, 2, 0, 2.5, 0;
0, 0.5, 0.5, 0.5, 1, 0.5, 1.5, 0.5, 2, 0.5, 2.5, 0.5;
0, 1, 0.5, 1, 1, 1, 1.5, 1, 2, 1, 2.5, 1;
0, 1.5, 0.5, 1.5, 1, 1.5, 1.5, 1.5, 2, 1.5, 2.5, 1.5;
0, 2, 0.5, 2, 1, 2, 1.5, 2, 2, 2, 2.5, 2;
0, 2.5, 0.5, 2.5, 1, 2.5, 1.5, 2.5, 2, 2.5, 2.5, 2.5]
Image zoom x2 using CV_32FC2
[ 10,  15,  20,  25,  30,  15;
35,  38,  40,  43,  45,  23;
60,  60,  60,  60,  60,  30;
50,  53,  55,  58,  60,  30;
40,  45,  50,  55,  60,  30;
20,  23,  25,  28,  30,  15]
Map CV_16SC2 for a zoom  X 2
Results[x0, y0, x1, y0, ...]
[0, 0, 0, 0, 1, 0, 1, 0, 2, 0, 2, 0;
0, 0, 0, 0, 1, 0, 1, 0, 2, 0, 2, 0 ...
more

If the source is not an image matrix, but a vector array consisting of detected feature points, can I still use remap() to rectify the array? In my application, I have already found the feature points through image processing of the original (distorted) image, so do not want to waste time to reprocessing the image. I just need to undistort and rectify the points in the vector array. Thanks.

( 2017-03-29 12:39:17 -0600 )edit

No remap is for image. you can use undistortPoints

( 2017-03-29 14:47:49 -0600 )edit