Using FLANN to match ORB descriptors
I'm am investigating methods on how to speed up an object tracking algorithm that uses local feature matching in each frame of the sequence. I think it would be attractive to use the FLANN kNN neighbor search since a database of descriptors is initialized in the first frame (ie database is not updated), and during every frame the new detected keypoints and descriptors are matched to the database, to attempt to find kNN for each descriptor. It would be attractive to build a tree during initialization and thus during each frame the kNN matching/searching should be significantly faster as we do not traverse the entire database for the given query descriptor's best match, as we have prior knowledge of the search space. However since ORB features are ideal when computing resources are limited, how does one use the FLANN matcher? I have read here that we should convert the descriptors to CV_32F, but also here that this does make sense for binary descriptors to float. What would be the ideal way to use the FLANN matcher with ORB descriptors?
#include "opencv2/highgui.hpp"
#include <opencv2/features2d/features2d.hpp>
#include "opencv2/core.hpp"
#include <cstdlib>
using namespace cv;
int main( int argc, char** argv )
{
Ptr<FeatureDetector> detector;
Ptr<DescriptorExtractor> descriptor;
detector = GFTTDetector::create();
descriptor = ORB::create();
FlannBasedMatcher fmatcher = FlannBasedMatcher(makePtr<flann::LshIndexParams>(12, 20, 2));
Mat img_1 = imread("/home/user/testA.jpg", IMREAD_GRAYSCALE );
std::vector<KeyPoint> keypointDatabase;
Mat database;
detector->detect(img_1, keypointDatabase);
descriptor->compute(img_1, keypointDatabase, database);
//Descriptors must not be converted to CV_32F
//Mat database32F;
//database.convertTo(database32F, CV_32F);
//fmatcher.add(database32F);
fmatcher.add(database);
fmatcher.train();
Mat img_1 = imread("/home/user/testB.jpg", IMREAD_GRAYSCALE);
std::vector<KeyPoint> keypoints;
Mat descriptors;
detector->detect(img_1, keypoints);
descriptor->compute(img_1, keypoints, descriptors);
std::vector<std::vector<DMatch> > matches;
fmatcher.knnMatchImpl(descriptors, matches, 2, cv::noArray(), false);
return 0;
}
However the error :
Unsupported format or combination of formats (type=5) in buildIndex_, file /home/user/opencv-3.4.0/modules/flann/src/miniflann.cpp, line 315
is produced when I called matcher.train(). What is the correct manner to build the tree in an initialization step?
just remove the (silly) float conversion, and you should be alright ?
That solves the problem thank you! But when I try to use the function matcher.knnMatchImpl(descriptors, matches, 2, cv::noArray(), false);, I get the error saying the : error: ‘struct cv::Ptr<cv::descriptormatcher>’ has no member named ‘knnMatchImpl’. Is it because it is a protected function? But i have declared a pointer to the class
please update your code above, then.
I have updated the code, the error produced is "error: ‘virtual void cv::FlannBasedMatcher::knnMatchImpl(cv::InputArray, std::vector<std::vector<cv::dmatch> >&, int, cv::InputArrayOfArrays, bool)’ is protected ". But it should not be as I have created a pointer to the class?