Building reconstruction using plane sweep
Hello everyone!
I'm new to OpenCV (only started this semester) although I already had some notions, mainly on Stereo Vision.
In a project I'm currently working on a project that reconstructs buildings from a set of images. However, I'm having some problems and I have some questions I hope someone can answer :D
What I'm doing is this:
1- Calibrate the camera using the tutorials example with a chessboard pattern. This gives me the intrinsic parameters matrix K and the distortion coefficients.
2- From a set of input images, I extract Canny edges and Hough lines. The Hough lines are the points where I expect to obtain better stereo matching among images.
3- Next, I try to obtain the extrinsic parameters for each image. For this, I iterate over all the images, find feature points and correspondences between image i and i-1 using a SurfFeatureDetector and a BruteForceMatcher. Find the fundamental matrix F and the Essential matrix E with: E = K^(t)FK. From E I can calculate the position and rotation of each image.
To this point I have all images positioned in space with the first image being at the origin and looking along positive Z.
4- Execute the Collins Plane Sweep using the positioned images and the Hough lines (interesting points). First all interesting points from all images are projected to a canonical plane (Z=z0) with the non linear planar homography:
Hi = K[r1 r2 z0*r3+t]
K is the intrinsic parameters matrix, r1, r2, r3 are the columns of the image rotation matrix and t is the translation matrix. Then for each sweep plane in the z axis (Z=zi) I use the points in the canonical plane to reproject them to this plane using:
xi = (zi-Cz)/(z0-Cz)*x0+(1-delta)*Cx
yi = (zi-Cz)/(z0-Cz)*y0+(1-delta)*Cy
(xi,yi) is the reprojected point in the plane Z=zi and [Cx Cy Cz] is the camera position. Then increment by one all plane cells inside a radius from such points.
In the end, all cells from each plane that have more than T votes are considered to contain a valid point in space, and as such I create a vertex there.
So my questions are:
I'm using an Nikon SLR Camera and I'm not sure if this is correct, since it has autofocus. However in the image tags, all images had a focal length of 18mm. I'm not sure whether it is the same focal length as the one in the intrinsic parameters.
In the matrix output by the calibration process (1) I get values of 10^3 order. In what metric are this values?
What is the purpose of the distortion coefficients and should I use them in my project? Where?
Am I calculating the Essential matrix and extrinsic parameters correctly? How can I be sure the results are good?
The building in the scene is about 20m away from the cameras, however I need a ...