Revision history [back]

Using FLANN to match ORB descriptors

I'm am investigating methods on how to speed up an object tracking algorithm that uses local feature matching in each frame of the sequence. I think it would be attractive to use the FLANN kNN neighbor search since a database of descriptors is initialized in the first frame (ie database is not updated), and during every frame the new detected keypoints and descriptors are matched to the database, to attempt to find kNN for each descriptor. It would be attractive to build a tree during initialization and thus during each frame the kNN matching/searching should be significantly faster as we do not traverse the entire database for the given query descriptor's best match, as we have prior knowledge of the search space. However since ORB features are ideal when computing resources are limited, how does one use the FLANN matcher? I have read here that we should convert the descriptors to CV_32F, but also here that this does make sense for binary descriptors to float. What would be the ideal way to use the FLANN matcher with ORB descriptors?

cv::FlannBasedMatcher matcher = cv::FlannBasedMatcher(cv::makePtr<cv::flann::LshIndexParams>(12, 20, 2));

database.convertTo(database32F, CV_32F);
matcher.add(database32F);
matcher.train();

However the error :

Unsupported format or combination of formats (type=5) in buildIndex_, file /home/user/opencv-3.4.0/modules/flann/src/miniflann.cpp, line 315

is produced when I called matcher.train(). What is the correct manner to build the tree in an initialization step?

Using FLANN to match ORB descriptors

cv::FlannBasedMatcher matcher #include "opencv2/highgui.hpp"
#include <opencv2/features2d/features2d.hpp>
#include "opencv2/core.hpp"
#include <cstdlib>
using namespace cv;

int main( int argc, char** argv )
{
    Ptr<FeatureDetector> detector;
    Ptr<DescriptorExtractor> descriptor;

    detector = cv::FlannBasedMatcher(cv::makePtr<cv::flann::LshIndexParams>(12, GFTTDetector::create();
    descriptor = ORB::create();
    FlannBasedMatcher fmatcher = FlannBasedMatcher(makePtr<flann::LshIndexParams>(12, 20, 2));

database.convertTo(database32F,     Mat img_1 = imread("/home/user/testA.jpg", IMREAD_GRAYSCALE );

    std::vector<KeyPoint> keypointDatabase;
    Mat database;
    detector->detect(img_1, keypointDatabase);
    descriptor->compute(img_1, keypointDatabase, database);

        //Descriptors must not be converted to CV_32F
        //Mat database32F;
        //database.convertTo(database32F, CV_32F);
matcher.add(database32F);
matcher.train();
        //fmatcher.add(database32F);

    fmatcher.add(database);
    fmatcher.train();

    Mat img_1 = imread("/home/user/testB.jpg", IMREAD_GRAYSCALE);

    std::vector<KeyPoint> keypoints;
    Mat descriptors;

    detector->detect(img_1, keypoints);
    descriptor->compute(img_1, keypoints, descriptors);

    std::vector<std::vector<DMatch> > matches;
    fmatcher.knnMatchImpl(descriptors, matches, 2, cv::noArray(), false);

    return 0;
}

However the error :

Unsupported format or combination of formats (type=5) in buildIndex_, file /home/user/opencv-3.4.0/modules/flann/src/miniflann.cpp, line 315

is produced when I called matcher.train(). What is the correct manner to build the tree in an initialization ~~step?~~ step?