Using FLANN to match ORB descriptors

asked 2018-08-10 06:06:25 -0500

gerardWalsh gravatar image

updated 2018-08-14 11:46:06 -0500

I'm am investigating methods on how to speed up an object tracking algorithm that uses local feature matching in each frame of the sequence. I think it would be attractive to use the FLANN kNN neighbor search since a database of descriptors is initialized in the first frame (ie database is not updated), and during every frame the new detected keypoints and descriptors are matched to the database, to attempt to find kNN for each descriptor. It would be attractive to build a tree during initialization and thus during each frame the kNN matching/searching should be significantly faster as we do not traverse the entire database for the given query descriptor's best match, as we have prior knowledge of the search space. However since ORB features are ideal when computing resources are limited, how does one use the FLANN matcher? I have read here that we should convert the descriptors to CV_32F, but also here that this does make sense for binary descriptors to float. What would be the ideal way to use the FLANN matcher with ORB descriptors?

#include "opencv2/highgui.hpp"
#include <opencv2/features2d/features2d.hpp>
#include "opencv2/core.hpp"
#include <cstdlib>
using namespace cv;

int main( int argc, char** argv )
    Ptr<FeatureDetector> detector;
    Ptr<DescriptorExtractor> descriptor;

    detector = GFTTDetector::create();
    descriptor = ORB::create();
    FlannBasedMatcher fmatcher = FlannBasedMatcher(makePtr<flann::LshIndexParams>(12, 20, 2));

    Mat img_1 = imread("/home/user/testA.jpg", IMREAD_GRAYSCALE );

    std::vector<KeyPoint> keypointDatabase;
    Mat database;
    detector->detect(img_1, keypointDatabase);
    descriptor->compute(img_1, keypointDatabase, database);

        //Descriptors must not be converted to CV_32F
        //Mat database32F;
        //database.convertTo(database32F, CV_32F);


    Mat img_1 = imread("/home/user/testB.jpg", IMREAD_GRAYSCALE);

    std::vector<KeyPoint> keypoints;
    Mat descriptors;

    detector->detect(img_1, keypoints);
    descriptor->compute(img_1, keypoints, descriptors);

    std::vector<std::vector<DMatch> > matches;
    fmatcher.knnMatchImpl(descriptors, matches, 2, cv::noArray(), false);

    return 0;

However the error :

Unsupported format or combination of formats (type=5) in buildIndex_, file /home/user/opencv-3.4.0/modules/flann/src/miniflann.cpp, line 315

is produced when I called matcher.train(). What is the correct manner to build the tree in an initialization step?

edit retag flag offensive close merge delete



just remove the (silly) float conversion, and you should be alright ?

berak gravatar imageberak ( 2018-08-11 00:59:54 -0500 )edit

That solves the problem thank you! But when I try to use the function matcher.knnMatchImpl(descriptors, matches, 2, cv::noArray(), false);, I get the error saying the : error: ‘struct cv::Ptr<cv::descriptormatcher>’ has no member named ‘knnMatchImpl’. Is it because it is a protected function? But i have declared a pointer to the class

gerardWalsh gravatar imagegerardWalsh ( 2018-08-11 10:17:33 -0500 )edit

please update your code above, then.

berak gravatar imageberak ( 2018-08-14 04:03:09 -0500 )edit

I have updated the code, the error produced is "error: ‘virtual void cv::FlannBasedMatcher::knnMatchImpl(cv::InputArray, std::vector<std::vector<cv::dmatch> >&, int, cv::InputArrayOfArrays, bool)’ is protected ". But it should not be as I have created a pointer to the class?

gerardWalsh gravatar imagegerardWalsh ( 2018-08-14 11:47:54 -0500 )edit