ORB match score and compare a Database
Hello everyone,
I know this question has already been asked but i can't find any examples actually.
I've got a database of cards (actually 3 but it will be over 20,000) and i need to find the best matching card from a webcam capture (Just like this video : https://www.youtube.com/watch?v=kbYDj...).
I think i've got a good set of matchs from a lot of topics but i can't figure out how to calculate a "score" or a percentage of "match" against 2 pictures.
I don't have access to any paid features of opencv (nonfree.hpp).
I'm pretty new at opencv and picture processing. If i can get a good score & a fast enought match algorithm, i'll pre-compute descriptors and put em in a database so i'll only have to compute the camera keypoints.
Here's some code of mine :
const array<string,3> pictures = {"D:\\visual\\test\\test\\Knotvine Paladin.xlhq.jpg",
"D:\\Visual\\test\\test\\Mordant Dragon.xlhq.jpg",
"D:\\Visual\\test\\test\\Shivan Hellkite.xlhq.jpg"};
void anotherTest( VideoCapture &capture )
{
Ptr<ORB> orb = ORB::create(300, 1.2f, 8, 31, 0, 4, ORB::FAST_SCORE, 31, 20);
BFMatcher matcher(NORM_HAMMING2, true);
vector<vector<KeyPoint>> objectskeypoints;
Mat descriptors;
vector<KeyPoint> keypoints;
vector<Mat> images;
vector<Mat> objects;
int index;
bool found;
// Create database keypoints & descriptors
for( index = 0; index < pictures.size(); ++index )
{
Mat img = imread(pictures[index], CV_LOAD_IMAGE_GRAYSCALE);
orb->detect(img, keypoints);
orb->compute(img, keypoints, descriptors);
objectskeypoints.push_back(keypoints);
objects.push_back(descriptors);
images.push_back(img);
}
Mat scene, grayScene;
Mat result;
Mat sceneDescriptors;
while( 1 )
{
capture >> scene;
Mat roi(scene, Rect(201, 75, 240, 340));
rectangle(scene, Point(201,75), Point(441,415), Scalar(255,0,0));
cvtColor(roi, grayScene, CV_BGR2GRAY);
CClock beginTime = CClock::GetCurrentClock();
orb->detect(grayScene, keypoints);
orb->compute(grayScene, keypoints, sceneDescriptors);
found = false;
index = 0;
for( index = 0; index < objects.size(); ++index )
//for( index = 0; index < 1; ++index )
{
//vector<vector<DMatch>> matchs1;
vector<DMatch> matchs1;
vector<DMatch> matchs2;
vector<DMatch> good_matchs, better_matchs, best_matchs, res_matchs;
try
{
Mat out, mask;
vector<KeyPoint> inliers1;
//matcher.knnMatch(objects[index], sceneDescriptors, matchs1, 2);
matcher.match(objects[index], sceneDescriptors, matchs1);
matcher.match(sceneDescriptors, objects[index], matchs2);
//getGoodMatchsFromknn(matchs1, good_matchs);
getGoodMatchsFromDistance(matchs1, better_matchs);
getGoodMatchsFromSymetry(better_matchs, matchs2, res_matchs);
}
catch( Exception &e )
{
string s = e.msg;
}
cout << res_matchs.size() << '/' << matchs1.size() << ' ' << ((double)res_matchs.size()*100/matchs1.size()) << ' ' << (CClock::GetCurrentClock() - beginTime).GetMilliSeconds() << "ms" << endl;
drawMatches(images[index], objectskeypoints[index], roi, keypoints, res_matchs, result, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
imshow("result", result);
waitKey(1);
}
}
}
// This is from the opencv tutorial
void getGoodMatchsFromDistance( std::vector<cv::DMatch> const& matchs, std::vector<cv::DMatch> &outMatchs )
{
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( vector<DMatch>::const_iterator it = matchs.begin(); it != matchs.end(); ++it )
{
double dist = it->distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
for( vector<DMatch>::const_iterator it = matchs.begin(); it != matchs.end(); ++it )
if( it->distance < 3*min_dist )
outMatchs ...
feature matching is meant to find the same scene in 2 different images, you're trying to abuse it for object classification, and it won't work.
Well, do you have an idea of how can i do to make this object classification ? I triied BoW but i can't make it work with orb. Even with some code examples that "should work" with ORB
no, BOW won't work with ORB features, you'd need float descriptors, like SIFT or SURF.