Difference between flann based matcher in C and C++?
I'm trying to use the C++ version of the flann based matcher, mainly so I can train it for several pictures ahead of time. The results however are not very satisfactory. As a little test I made a program with two implementations, one using the C API and the other using the C++ API. For some reason the C implementation finds 50% more good matches than the C++ version. And when I use the train() function, the results are even worse. Not only is the number of good matches even lower, the quality of the matches is also a lot worse. (Which becomes obvious when I draw them.)
It seems the difference must be with the flann code, as the number of SURF descriptors generated by either method are equal.
The (standalone) code is below. I also attached the images I used to test this here: C:\fakepath\Cow.jpg C:\fakepath\Test.jpg
Any explanation of the different results would be greatly appreciated.
#include <stdio.h>
#include <iostream>
#include <cv.hpp>
#include "opencv2/core/core.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/nonfree/features2d.hpp"
using namespace cv;
using namespace std;
#define HESSIAN_VALUE 1500
#define THRESHOLD 0.6
void readme();
int test1(int argc, char *argv[])
{
Mat img_1 = imread( argv[1], IMREAD_GRAYSCALE );
Mat img_2 = imread( argv[2], IMREAD_GRAYSCALE );
vector<KeyPoint> keypoints1;
Mat descriptors1;
vector<KeyPoint> keypoints2;
Mat descriptors2;
vector< vector< DMatch > > matches;
vector< DMatch > good_matches;
SurfFeatureDetector detector(HESSIAN_VALUE);
detector.detect(img_1, keypoints1);
detector.detect(img_2, keypoints2);
SurfDescriptorExtractor extractor;
extractor.compute(img_1, keypoints1, descriptors1);
extractor.compute(img_2, keypoints2, descriptors2);
FlannBasedMatcher flannMatcher (new cv::flann::KDTreeIndexParams(4), new cv::flann::SearchParams(64));
// vector<Mat> descriptorList;
// descriptorList.push_back(descriptors1);
// flannMatcher.add(descriptorList);
// flannMatcher.train();
// flannMatcher.knnMatch(descriptors2, matches, 2);
flannMatcher.knnMatch(descriptors1, descriptors2, matches, 2);
for (int i = 0; i < keypoints1.size(); ++i)
{
if (matches[i].size() < 2)
continue;
const DMatch &m1 = matches[i][0];
const DMatch &m2 = matches[i][1];
if (m1.distance <= THRESHOLD * m2.distance)
good_matches.push_back(m1);
}
std::cout << "1- matches found: " << good_matches.size() << std::endl;
std::cout << "1- descriptors1 found: " << descriptors1.size() << std::endl;
std::cout << "1- descriptors2 found: " << descriptors2.size() << std::endl;
Mat img_matches;
drawMatches( img_1, keypoints1, img_2, keypoints2, good_matches, img_matches );
imshow("Matches", img_matches );
return 0;
}
int test2(int argc, char *argv[])
{
IplImage* img_1 = cvLoadImage( argv[1], CV_LOAD_IMAGE_GRAYSCALE );
IplImage* img_2 = cvLoadImage( argv[2], CV_LOAD_IMAGE_GRAYSCALE );
CvSeq * keypoints1 = 0;
CvSeq * descriptors1 = 0;
CvSeq * keypoints2 = 0;
CvSeq * descriptors2 = 0;
CvMemStorage* storage = cvCreateMemStorage(0);
CvSURFParams params = cvSURFParams(HESSIAN_VALUE, 1);
cvExtractSURF( img_1, 0, &keypoints1, &descriptors1, storage, params );
cvExtractSURF( img_2, 0, &keypoints2, &descriptors2, storage, params );
int length = (int)(descriptors1->elem_size/sizeof(float));
cv::Mat m_object(descriptors1->total, length, CV_32F);
cv::Mat m_image(descriptors2->total, length, CV_32F);
vector<int> good_matches;
// copy descriptors
CvSeqReader obj_reader;
float* obj_ptr = m_object.ptr<float>(0);
cvStartReadSeq( descriptors1, &obj_reader );
for(int i = 0; i < descriptors1->total; i++ )
{
const float* descriptor = (const float*)obj_reader.ptr;
CV_NEXT_SEQ_ELEM( obj_reader.seq->elem_size, obj_reader );
memcpy(obj_ptr, descriptor, length*sizeof(float));
obj_ptr += length;
}
CvSeqReader img_reader;
float* img_ptr = m_image.ptr ...
test2 seems like a mixture of C and C++, anyway, have you tried to use
cv::flann::Index flann_index(m_image, cv::flann::KDTreeIndexParams(4)); flann_index.knnSearch(m_object, m_indices, m_dists, 2, cv::flann::SearchParams(64) );
in test1? It would be very strange if this gives better results than FlannBasedMatcher.
It's true it's a bit of a mix of C and C++. The flann part is pretty much a straight copy of the flannFindPairs() function from the find_object.cpp sample code in openCV. I call it the C version because it uses CvSeq, IplImage and cvExtractSURF rather than their C++ counterparts. When I do as you suggest, indeed the results are much better than FlannBasedMatcher. I have added a test3() function to show it. This makes me wonder if I'm using the FlannBasedMatcher in the wrong way (but then, so do all the examples I found on the web) and I tried reversing the parameters descriptors1 and descriptors2 in the knnMatch() call. Interestingly, that gave me the exact same bad results as using the (commented) lines of code using the train() function. Something seriously wrong with FlannBasedMatcher?
I found a mistake that overstated the result. Rather than finding three times as many good matches it's actually one-and-half times as many. Still a very big difference and still the same problem with training. I have edited the original post to improve it.