Revision history [back]

I try to match images against a database (100 images) to find the best matched image.

With OpenCV, I first tried SIFT feature with FlannBasedMatcher and found it worked well. But, because SIFT is patented, I turned to BRISK and FREAK. For both, I used BRISK detector (i.e. AGAST) and BruteForceMachter as they are binary descriptors. As claimed in the original BRISK and FREAK papers, they should have comparable accuracy and be faster than SIFT.

But unfortunately the result was really bad. My code of matching part looks like

BruteForceMatcher<Hamming> matcher;

//before matching, first add all descriptors of 100 images into matcher
...(get train_descriptors out of each image)
matcher.add(vector<Mat>(1, train_descriptors));
...(done.)

vector<KeyPoint> key_points;
Mat descriptors;
BRISK detector(30, 3, 1.0f);
detector.detect(test_image, keypoints);
FREAK extractor; //or BRISK extractor(30, 3, 1.0f);
extractor.compute(test_image, keypoints, descriptors);

vector<DMatch> matches;
matcher.match(descriptors, matches);
//choose good matches
vector<DMatch> good_matches;
for(size_t i=0;i<matches.size();i++) {
    if(matches[i].distance < 90) {
         good_matches.push_back(matches[i]);
    }
}

I used a matching Hamming distance threshold of 90 as suggested in the BRISK paper. However, the good_matches contains not so many matches (usually 20 matches or so), out of which only about 25% (or even worse) belong to the correct matched image. This result is much much worse than SIFT where this proportion is commonly over 90%.

In addition, they were not faster either. After the train set (100 images) is trained and indexed using FlannBasedMatcher, the matching of SIFT is actually much faster than BRISK/FREAK with BruteForceMatcher. Therefore, I also tried to use FlannBasedMatcher with LshIndexParams for BRISK/FREAK like

FlannBasedMatcher matcher(new flann::LshIndexParams(20,10,2));

But the results were still bad and not faster either.

So I have two questions here:

1. How to determine a good match (to exclude outliers) for binary descriptors?

2. Which matcher should be used for FREAK/BRISK to speed up the matching? (comparable to SIFT with FlannBasedMatcher).

Thank you in advance for any hints.

I try to match images against a database (100 images) to find the best matched image.

With OpenCV, I first tried SIFT feature with FlannBasedMatcher and found it worked well. But, because SIFT is patented, I turned to BRISK and FREAK. For both, I used BRISK detector (i.e. AGAST) and BruteForceMachter as they are binary descriptors. As claimed in the original BRISK and FREAK papers, they should have comparable accuracy and be faster than SIFT.

But unfortunately the result was really bad. My code of matching part looks like

BruteForceMatcher<Hamming> matcher;

//before matching, first add all descriptors of 100 images into matcher
...(get train_descriptors out of each image)
matcher.add(vector<Mat>(1, train_descriptors));
...(done.)

vector<KeyPoint> key_points;
Mat descriptors;
BRISK detector(30, 3, 1.0f);
detector.detect(test_image, keypoints);
FREAK extractor; //or BRISK extractor(30, 3, 1.0f);
extractor.compute(test_image, keypoints, descriptors);

vector<DMatch> matches;
matcher.match(descriptors, matches);
//choose good matches
vector<DMatch> good_matches;
for(size_t i=0;i<matches.size();i++) {
    if(matches[i].distance < 90) {
         good_matches.push_back(matches[i]);
    }
}

I used a matching Hamming distance threshold of 90 as suggested in the BRISK paper. However, the good_matches contains not so many matches (usually 20 matches or so), out of which only about 25% (or even worse) belong to the correct matched image. This result is much much worse than SIFT where this proportion is commonly over 90%.

In addition, they were not faster either. After the train set (100 images) is trained and indexed using FlannBasedMatcher, the matching of SIFT is actually much faster than BRISK/FREAK with BruteForceMatcher. Therefore, I also tried to use FlannBasedMatcher with LshIndexParams for BRISK/FREAK like

FlannBasedMatcher matcher(new flann::LshIndexParams(20,10,2));

But the results were still bad and not faster either.

So I have two questions here:

1. How to determine a good match (to exclude outliers) for binary descriptors?

2. Which matcher should be used for FREAK/BRISK to speed up the matching? (comparable to SIFT with FlannBasedMatcher).

Thank you in advance for any hints.