Evaluation of binary descriptors

asked 2013-06-09 07:30:20 -0600

Hi,

I'm using the project (EXAMPLE) detector_descriptor_matcher_evaluation to make a comparison of the performance of binary descriptors.

The results that I get basically show that in cases of blur, exposure and jpeg compression SIFT > BRIEF > ORB > BRISK > FREAK > SURF and in viewpoint changes and rotation+zoom SIFR>FREAK>BRIEF>BRISK>SURF>ORB where ">" denotes "outperforms".

Those results are a bit contradicting of my intuition. ORB is an "improved" version of BRIEF where learned pairs and rotation invariance is added, so how come BRIEF outperforms ORB?

In addition, I'm getting very bad results for the "zoom and rotation tests" (bark and boat datasets") – lower than 2% precision for all the image pairs other than (im1,im2). Should I expect such low precision?

Thanks in advance,

Gil.

answered 2013-06-09 08:16:37 -0600

Notas

362 ●8

How much is each of those algorithms better than the ones before? Might be interesting to know. Anyway, the big problem is the matching. It's ineffective. VERY!

First off, use KNN-matching with two nearest neighbours. Then do the ratiotest, proposed by Lowe, to eliminate possible bad matches. Then use the symmetrytest / crosscheck to further remove bad matches. Lastly, use the epipolargeometry to find the best possible matches. For code, see the OpenCV-cookbook, the class with the code of the tests I wrote about in there is called RobustMatcher.

Simply finding the best match for each keypoint without regarding the geometric relation of the two images and the matches might work if you have VERY distinct keypoints and therefore descriptors, but if you add noise and other changes, it gets hard. And then a simple "find the best match for each keypoint" is not enough.

edit flag offensive delete link

Comments

Hi,

Thanks for the response. I think that since I'm performing an evaluation of the descriptors distinctiveness ability, I shouldn't use the geometric relation. Of course that it is vital in order to get good correspondences, but that is not the purpose of the evaluation. However, I will use the other suggestion you made.

Thanks again! Gil.

GilLevi ( 2013-06-09 10:26:41 -0600 )edit

add a comment

Evaluation of binary descriptors

1 answer

Comments

Links

Question Tools

Stats

Related questions

Evaluation of binary descriptors edit

1 answer

Comments

Links

Question Tools

Stats

Related questions

Evaluation of binary descriptors