Jacek's profile - activity

Some time ago I've implemented 5-pt algorithm (according to Nister solution: see An Efficient Solution to the Five-Point Relative Pose Problem). The code uses additional packages: Eigen (for singular value decomposition) and rpoly.cpp (Jenkins-Traub real roots finder) and has about 500 lines of code (part of the code was generated in Matlab).

Experiments with synthetic data showed that it's more accurace and less sensitive to noise than 7-pt or 8-pt algorithms. And, as theory says, doesn't suffer from planar degeneracy.

If you are interested I can share a source code so it can be modified and added to OpenCV.

Maybe I'm missing something but it's not possible to perform metric reconstruction from 2 uncalibrated images (unless you make some additional assumptions, e.g. you know/estimate location of a vanishing point or you known something about camera calibration). You can make reconstruction up to a projective transformation. See projective reconstruction theorem in Harley book.

How do you calculate camera matrices from fundamental matrix? What formula are you using? Without knowing camera intrinsic matrix it's not possible. Are you really sure this step is correct?

triangulatePoints returns homogeneous coordinates of 3D points, one point in a column.

Homogeneous coordinate(X,Y,Z,W) = euclidean coordinate (X/W, Y/W, Z/W).

To filter out incorrect your matches you can use epipolar constraint. Use findFundamentalMat function with FM_RANSAC estimation method to estimate fundamental matrix based on matching points from 2 images. RANSAC can deal with situation when you have many outliers.

The function returs estimated fundamental matrix and a vector indicating which pair of matching points is inlier and which is outlier.

I think boardSize is incorrect in your code. boardSize should be equal to a number of internal corners, so in your case it should be cv::Size boardSize(5,4).

Can you please more specific? What is the matrix you refer to? Is it a fundamental matrix for 2 images?

Camera matrix = [fx 1 cx; 1 fy cy;0 0 1] is wrong it should be [fx 0 cx; 0 fy cy;0 0 1] (or [fx s cx; 0 fy cy;0 0 1] if you have a camera with non-zero skew s but this is seldom the case).

Besides the procedure you described looks OK. Only reprojection error = 10 is huge, I'm not sure if this is for purpose but you usually should use reprojection error like 1 or 2 pixels.

I think it's a normal behaviour that region with x-coordinates between 0 and max_disparity is not reconstructed.

Suppose on a rectified left image you have a point with x-coordinate x0, where 0 < x0 < max_disparity. Potential matches for this point on the right rectified image have x-coordinate in < x0-max_disparity; x0 > range. But if x0 < max_disparity part of this range has negative x-coordinate and is not visible on the rectified right image. So disparity cannot be calculated.

And one advice: for better results use StereoSGBM algorithm. It usually gives much better results than simple block matching (StereoBM) algorithm.

The reason of your issue could be that disparity computed by SGBM is for some reason multiplied by 16.

See this excerpt from OpenCV documentation:

disp – Output disparity map. It is a 16-bit signed single-channel image of the same size as the input image. It contains disparity values scaled by 16. So, to get the floating-point disparity map, you need to divide each disp element by 16.

Distance to the camera (Z coordinate) is inversly proportional to disparity. So you can equivalently multiply Z-coodinate by 16. Taking into account this scaling coefficient you have: 6*16 = 96 which is pretty close to 100.

I don't think any OpenCV stereo algorithms allows to use more than 1 pair of images. StereoSGBM is designed to work with 2 images only. I'm also not aware of any available stereo software that works with many image pairs taken from the same locations but with varying lighting conditions.

Potentially SGBM can be changed to use more than 1 pair of images (assuming all pairs are taken from the same camera positions). One step in SGBM algorithm is to compute correspondence score between a block of pixels on the left image with a block of pixels on the right. It should be sufficient to change only this step of the algorithm, so that correspondence between blocks of pixel on each image pair is taken into account. And it can be good research topic to find out what's the best way to combine correspondence scores from many image pairs (e.g. take minimum score, or maybe sum of scores from each image pair?). But this requires some code change on SGBM algorithm.

There's no a simple single function that will give you camera intrinsic parameters. Calibration process requires a few steps:

1. Acquisition of calibration images (with chessboard pattern or with circle pattern)
2. Detecting chessboard corners(or blobs in case of circle pattern) with subpixel accuracy
3. Finding camera calibration paramters: intrinsic matrix and distortion coefficients with

The easiest approach is to use an example provided with OpenCV - calibration.exe. You can use it with your own images, so it'll do a calibration for you and give you camera intrinsic matrix and distortions coefficients.

Read below tutorial, it explains how OpenCV calibration example works and how to use it with your own calibration images: http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html#cameracalibrationopencv

StereoSGBM() is in C++ API (since version 2.3 AFAIK). See: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html?highlight=stereosgbm#StereoSGBM

If distortion is not too big you may still use ICP. ICP doesn't require that 2 point clouds are exactly identical. One point cloud may contain points corrupted with a noise and ICP will still work.

The catch is that ICP requires point clouds to be roughly aligned. So if corresponding points in your point clouds are quite close you may use ICP. Otherwise you'll need to use some method to roughly align your point clouds (e.g. compute some kind of feature descriptors for each point, use these feature descriptors to create matching between points in both clouds and roughly align point clouds using these matchings) and then use ICP for final registration.

It seems GraphCut stereo algorithm it's not included in the latest OpenCV version (at least I wasn't able to find anything in the documentation).

I can suggest using StereoSGBM(). Semi Global Block Matching stereo algorithm gives very good results (similar to best global methods, like GraphCut) but at the speed comparable to simple local methods (like block matching algorithms). I've tried it myself and reasults were really very good.

OpenCV algorithms for computing stereo correspondence (StereoSGBM, StereoBM) require rectified images. If you use them on non-rectified images they won't work correctly.

Rectification is a transformation of images such that projections of the same scene point have equal y-coordinate on both images. E.g. if a projection of some 3D point onto a left image has y-coordinate = 150 it'll have y-coordinate = 150 on the second image. So rectification simplifies finding corresponding points (and subsequently simplifies computation of a depth map).

If you want to build your stereo vision application use the following approach:

1. Calibrate cameras. The simplest approach is to use calibration procedure (stereo_calib.cpp/stereo_calib.exe) distributed in OpenCV examples. This will give intrinsic and extrinsic camera parameters. From rectification procedure you should get reprojection error on the level of 0.5 pixel. Much bigger reprojection error (above 1 pixel) usually means that there're some issues with a calibration.
2. Use stereo camera parameters estimated by stereo calibration procedure to rectify ALL images. You may verify correctness of this step by looking on rectified images: projections of the same scene points on both images should have the same y-coordinate.
3. Use stereo correspondence function on RECTIFIED images. I can suggest using StereoSGBM - it usually produces better results than simple StereoBM procedure.

Function cvInitUndistortRectifyMap needs to be used only once - after you get stereo camera parameters from stereo calibration procedure. This funciton computes a mapping needed to rectify images. Then you rectify ALL images using mapping returned by cvInitUndistortRectifyMap.

I don't think RANSAC is a good idea in your case. RANSAC can be used when you have a number of measurements (e.g. pairs of corresponding points from 2 sets) containing some outliers (e.g. incorrectly matched points). The more outliers you have the more RANSAC iterations are needed to estimate parameters with a given confidence. In your scenario, choosing matching points randomly, you'll need a really HUGE number of iterations to ensure proper matching is found. Computational cost may be prohibitive.

Can you give more details about your problem. Why are you trying to estimate perspective transform? Are these 2D points projections of some 3D points?

I've found the following issue on bug tracker: SURF : different number of keypoints between OpenCV 2.3 & 2.4 (Bug #1911)

It seems that there are some differences in SURF feature detector in OpenCV 2.4. One change is that extended parameter is set to true by default, so descriptor length is 128 not 64. Also it seems that in 2.4 more keypoints are located - but if this is for purpose or is some error/bug is not clear yet.

See: http://code.opencv.org/issues/1911