# Revision history [back]

### Building reconstruction using plane sweep

Hello everyone!

I'm new to OpenCV (only started this semester) although I already had some notions, mainly on Stereo Vision.

In a project I'm currently working on a project that reconstructs buildings from a set of images. However, I'm having some problems and I have some questions I hope someone can answer :D

What I'm doing is this:

1- Calibrate the camera using the tutorials example with a chessboard pattern. This gives me the intrinsic parameters matrix K and the distortion coefficients.

2- From a set of input images, I extract Canny edges and Hough lines. The Hough lines are the points where I expect to obtain better stereo matching among images.

3- Next, I try to obtain the extrinsic parameters for each image. For this, I iterate over all the images, find feature points and correspondences between image i and i-1 using a SurfFeatureDetector and a BruteForceMatcher. Find the fundamental matrix F and the Essential matrix E with: E = K^(t)FK. From E I can calculate the position and rotation of each image.

To this point I have all images positioned in space with the first image being at the origin and looking along positive Z.

4- Execute the Collins Plane Sweep using the positioned images and the Hough lines (interesting points). First all interesting points from all images are projected to a canonical plane (Z=z0) with the non linear planar homography:

Hi = K[r1 r2 z0*r3+t]


K is the intrinsic parameters matrix, r1, r2, r3 are the columns of the image rotation matrix and t is the translation matrix. Then for each sweep plane in the z axis (Z=zi) I use the points in the canonical plane to reproject them to this plane using:

xi = (zi-Cz)/(z0-Cz)*x0+(1-delta)*Cx
yi = (zi-Cz)/(z0-Cz)*y0+(1-delta)*Cy


(xi,yi) is the reprojected point in the plane Z=zi and [Cx Cy Cz] is the camera position. Then increment by one all plane cells inside a radius from such points.

In the end, all cells from each plane that have more than T votes are considered to contain a valid point in space, and as such I create a vertex there.

So my questions are:

I'm using an Nikon SLR Camera and I'm not sure if this is correct, since it has autofocus. However in the image tags, all images had a focal length of 18mm. I'm not sure whether it is the same focal length as the one in the intrinsic parameters.

In the matrix output by the calibration process (1) I get values of 10^3 order. In what metric are this values?

What is the purpose of the distortion coefficients and should I use them in my project? Where?

Am I calculating the Essential matrix and extrinsic parameters correctly? How can I be sure the results are good?

The building in the scene is about 20m away from the cameras, however I need a square plane with a side of 210^8 and each cell is 110^5 for the plane sweep algorithm. This is extremely large and yields a large amount of points ranging from

 min=[-9.995e+07, -9.995e+07, 0]


to

 max=[9.995e+07, 9.995e+07, 36500]


the z values in these vectors are the sweep planes and the value is correct (it is generated by me). However, I'm not so sure about the x and y values.

The collins plane sweep also has a statistical model of clutter to reduce the number of points to only the ones that are interesting. However I did not understand that part yet :S

Is there any plane sweep implementation available I can take a look at?

Sorry for the long post, but I'm not sure if I'm doing things right.