# object detection using SURF & FLANN

hi!

i write one program for finding one object in a image file in a video . i read the istruction in the opencv manual and i use the SURF and the FLANN matcher. the program draw lines between the corners in the mapped object in the video file. the problem is that in some frame the lines are not present or lines draw a strange figure.

for example...this is the correct output

and this is the wrong result for some frame

this is the code for the program:

void surf_detection(Mat img_1,Mat img_2); /** @function main */

int main( int argc, char** argv )
{

int i;
int key;

CvCapture* capture = cvCaptureFromAVI("video.MPG");// Read the video file

if (!capture){

cout <<" Error in capture video file" << endl;
return -1;
}

CvVideoWriter *writer = 0;
int isColor = 1;

//Get capture device properties
cvQueryFrame(capture); // this call is necessary to get correct
// capture properties
int frameH    = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT);
int frameW    = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH);
int fps       = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FPS);
int numFrames = (int) cvGetCaptureProperty(capture,  CV_CAP_PROP_FRAME_COUNT);

writer=cvCreateVideoWriter("out.avi",CV_FOURCC('P','I','M','1'),
fps,cvSize(frameW,frameH),isColor);

IplImage* img = 0;

for(i=0;i<numFrames;i++){
cvGrabFrame(capture);          // capture a frame
img=cvRetrieveFrame(capture);  // retrieve the captured frame

printf("Frame n° %d \n", i );

surf_detection (img_template,img);

cvWriteFrame(writer,img);      // add the frame to the file
cvShowImage("mainWin", img);
key=cvWaitKey(20);           // wait 20 ms

}
cvReleaseVideoWriter(&writer);

//waitKey(0); // Wait for a keystroke in the window
return 0;
}

/** @function surf_detection */
/** USE OF FUNCTION*/
/** param1: immagine del template*/
/** param2: immagine estratta dal video*/
void surf_detection(Mat img_1,Mat img_2)
{

if( !img_1.data || !img_2.data )
{
std::cout<< " --(!) Error reading images " << std::endl;

}

//-- Step 1: Detect the keypoints using SURF Detector
int minHessian = 400;
SurfFeatureDetector detector( minHessian );
std::vector<KeyPoint> keypoints_1, keypoints_2;

std::vector< DMatch > good_matches;

do{

detector.detect( img_1, keypoints_1 );
detector.detect( img_2, keypoints_2 );

//-- Draw keypoints

Mat img_keypoints_1; Mat img_keypoints_2;
drawKeypoints( img_1, keypoints_1, img_keypoints_1, Scalar::all(-1), DrawMatchesFlags::DEFAULT );
drawKeypoints( img_2, keypoints_2, img_keypoints_2, Scalar::all(-1), DrawMatchesFlags::DEFAULT );

//-- Step 2: Calculate descriptors (feature vectors)
SurfDescriptorExtractor extractor;
Mat descriptors_1, descriptors_2;
extractor.compute( img_1, keypoints_1, descriptors_1 );
extractor.compute( img_2, keypoints_2, descriptors_2 );

//-- Step 3: Matching descriptor vectors using FLANN matcher
FlannBasedMatcher matcher;
std::vector< DMatch > matches;
matcher.match( descriptors_1, descriptors_2, matches );
double max_dist = 0;
double min_dist = 100;

//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_1.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist )
min_dist = dist;
if( dist > max_dist )
max_dist = dist;
}

printf("-- Max dist : %f \n", max_dist );
printf("-- Min dist : %f \n", min_dist );

//-- Draw only "good" matches (i.e. whose distance is less than 2*min_dist )
//-- PS.- radiusMatch can also be used here.

for( int i = 0; i < descriptors_1.rows; i++ )
{
if( matches[i].distance < 2*min_dist )
{
good_matches.push_back( matches[i]);
}
}

}while(good_matches.size()<100);

//-- Draw only "good" matches
Mat img_matches;
drawMatches( img_1, keypoints_1, img_2, keypoints_2,good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );

//-- Localize the ...
edit retag close merge delete

Hi Andrea, from which university do you come from?

( 2012-09-01 11:41:26 -0500 )edit

Sort by » oldest newest most voted

Hi, It all depends on your computation time constraints.

Whatever the descriptor (SIFT, SURF, etc.)... each one having its computation time...

Using the FLANN matcher is fast but is approximate so that mismatches happen more often... then, it is common to see more mismatches even if using an accurate descriptor only because the matcher is not that appropriate... then, find the compromise !

Instead of using the FlannBasedMatcher, you should try C++: BFMatcher::BFMatcher(int normType, bool crossCheck=false ) http://docs.opencv.org/modules/features2d/doc/common_interfaces_of_descriptor_matchers.html?highlight=crosscheck.

This matcher is brute force then it requires much more processing power but... it is more precise.

In addition, you can try different setups : choose the distance (normType parameter) : NORM_L1 is efficient and low cost, NORM_L2 "can" be better but is more expensive.. but you can use NORM_L2SQR which is equivalent to the L2 distance but avoids the square root computation, avoiding some CPU usage.

If you use crossCheck=true, this will allow a more refined matching by trying to match in both directions (img1->img2 and img2->img1) and keep common matches... but this will result in fewer matches. However, take care, a bug report has been sent this summer : on current trunk, crossCheck=true leads to some wrong computations... (see http://code.opencv.org/issues/2292

If you do not use the crossCheck=true setup, then, I recommend to not use directly the BFmatchers object but prefer the static allocation method :

C++: Ptr<descriptormatcher> DescriptorMatcher::create(const string& descriptorMatcherType), then, use "BruteForce-L1", "BruteForce-L2", "BruteForce-SL2" or "FlannBased" to choose the appropriate matching method, this makes your program more flexible.

Have a nice code ;o)

more

nice answer +1! didn't know about NORM_L2SQR. does it corrispond to BruteForce-SL2 ?

( 2012-09-01 11:44:51 -0500 )edit

Hello, Yes, NORM_L2SQR and BruteForce-SL2 seem to do the same job bit are used from different areas (respectively BF_matcher constrcutor and DescriptorMatcher::create method). See ya

( 2012-09-02 08:04:57 -0500 )edit
1

BFMatcher is also the matcher to use with new FREAK descriptor, using NORM_HAMMING as type because it is a binary descriptor.

( 2012-09-20 03:04:05 -0500 )edit

I've had similar problems. The matching algorithm tries to find a match for every point in the first image. If there's more keypoints in the first image than there is in the second image, this means, that some keypoints in the second image will be matches to more than one keypoint in the first. (This seems to be the case in your examples, too).

In extreme cases (just a few keypoints in the second image) RANSAC then sometimes finds a homography that transforms every source point to a single destination point. Try to compute the matches by having the image with the fewer keypoints in the first position. You will get less but better matches.

Note: when swapping the image descriptors for matching, you will have to invert the homography matrix.

more

Nice, i think that might also help.

( 2012-08-31 08:24:48 -0500 )edit

Your used reference image is probably not very good for feature detection. It consists of large homogeneous regions and repetitive patterns.

But maybe some outlier rejection could help, e.g. a ratio test: Reject matches if distance of best match/distance of second best match > 0.8 (best match = nearest neighbor).

If results are still bad, maybe an approach with hough transformation (detection of straight lines) would be better.

more

The wrong results mean that somehow the matches were not very good so the resulting homography estimated based on those matches does not make sense. You have to somehow improve your matches, but that is easier said than done. You could start by trying to use ORB instead of SURF, it might give you better results. At least it will certainly be faster to process.

EDIT:

Try to be less strict on what you consider to be "good matches". That would be this line:

if( matches[i].distance < 2*min_dist )


This means that findHomography will have more points to use: processing will be slower but your chances of having a good homography might increase.

more

2

You could also try ASIFT, which they claim is really good http://code.opencv.org/projects/opencv/wiki/2012#2012-07-24 (See Vadim's note). Although, I think is yet to be implemented in OpenCV...

( 2012-08-01 05:41:11 -0500 )edit

According to those notes ASIFT is not yet implemented in OpenCV. But FREAK is, it is also an alternative to SURF but I have not tried it yet :)

( 2012-08-01 05:59:22 -0500 )edit

[andrea] I have tried to do some debug in my code and I have verified that the Homography matrix H is wrong for some frames! the exact H is:

H = [0.6487030067242824, -0.08631426465278198, 261.6879493611227; 0.01335918943588019, 0.7381261562725441, 68.20078368844314; -0.0001412237402441972, -5.520086932186027e-06, 1]

and for some frames it is:

H = [-2.57857982218806, 0.3892895299010169, 394.0872195148755; -0.7676881270794974, 0.2898285736196523, 107.9234724910376; -0.006545151477964493, 0.0009948206740805412, 1]

now I'm going to try your method, but I am not sure that I can change the SURF with the ORB... in this case some one know a method for correct the wrong Homography?

( 2012-08-03 16:12:40 -0500 )edit

@Kirill: see my answer. I'm pretty sure this is the reason. Maybe the matching algorithm should be improved by only using matches for every descriptor in the image with the fewer keypoints?

( 2012-08-31 08:10:40 -0500 )edit

Official site

GitHub

Wiki

Documentation