How to reduce image descriptor dimension to speed up image retrival [closed]

asked 2018-08-02 02:51:07 -0600

This is a image retrival example, get image sift descritprs which is n rows 128 cols, then call flann function to match, I want to speed it up, so i use PCA to reduce vector dimension such as 64 cols, but after PCA it can't find any match, Is then PCA code is wrong or can't call flann function to match after PCA processing

code:


#include <stdio.h>
#include <unistd.h>
#include <iostream>

#include <opencv2 opencv.hpp="">
#include <opencv2 xfeatures2d.hpp="">

using namespace cv;
using namespace cv::xfeatures2d;
using namespace std;

Mat PcaDecColumn(const Mat &src, int dstColNum)
{
    Mat beforePCA = src.clone();

    PCA pca(beforePCA, Mat(), CV_PCA_DATA_AS_ROW, dstColNum);
    Mat afterProject = pca.project(beforePCA);
    //Mat dst = pca.backProject(afterProject);
    Mat dst = afterProject;
    return dst;
}

    int
main( int argc, char **argv )
{
    cv::Mat queryImage = cv::imread(argv[1]);
    cv::Mat frameImage = cv::imread(argv[2]);
    if (frameImage.empty() || queryImage.empty())
    {
        printf( "%s %d err\n", __func__, __LINE__ );
        return -1;
    }

    vector<keypoint> queryKeypoints;
    vector<keypoint> frameKeypoints;
    Mat queryDescriptors;
    Mat frameDescriptors;

    //Ptr<orb> orb = ORB::create(1000);
    Ptr<feature2d> orb = xfeatures2d::SIFT::create();
    orb->detectAndCompute(queryImage, noArray(), queryKeypoints, queryDescriptors);
    orb->detectAndCompute(frameImage, noArray(), frameKeypoints, frameDescriptors);

    queryDescriptors = PcaDecColumn(queryDescriptors, 64);
    frameDescriptors = PcaDecColumn(frameDescriptors, 64);

    if (frameDescriptors.empty() || queryDescriptors.empty())
    {
        printf( "%s %d err\n", __func__, __LINE__ );
        return -1;
    }

    vector<dmatch> matches;
    vector<vector<dmatch> > totalMatches;
    FlannBasedMatcher matcher;
    matcher.knnMatch(frameDescriptors, queryDescriptors, totalMatches, 2);

    const float minRatio = 1.f / 1.5f;
    for (size_t i = 0; i < totalMatches.size(); i++) {
        const DMatch& bestMatch = totalMatches[i][0];
        const DMatch& betterMatch = totalMatches[i][1];

        float  distanceRatio = bestMatch.distance / betterMatch.distance;
        if (distanceRatio < minRatio)
            matches.push_back(bestMatch);
    }

    printf( "matches size:%d\n", matches.size() );

    Mat img_match;
    drawMatches(frameImage, frameKeypoints, queryImage, queryKeypoints, matches, img_match);
    imshow("match", img_match);
    waitKey(0);

    return 0;
}

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by tczengming
close date 2018-08-07 02:50:49.539398

Comments

if at all, you should make a single PCA offline (maybe even from independant data) and use that to project both the train & the query data(again, using the SAME projection matrix), else you're comparing apples to pears in the end.

but maybe you could just use the 64bit SURF (with UPRIGHT flag) ? it's also almost as fast as ORB !

berak gravatar imageberak ( 2018-08-02 03:11:55 -0600 )edit

since you cant use the above feature matching in a useful way for CBIR like tasks (it would need a "global" image descriptor, and clustering/classification based on that),

please also lookup "BagOfWords" and "vlad descriptors"

berak gravatar imageberak ( 2018-08-02 03:26:26 -0600 )edit

tks break gravatar image break, this is new code:
// modify here PCA pca(queryDescriptors, Mat(), CV_PCA_DATA_AS_ROW, col); queryDescriptors = pca.project(queryDescriptors); frameDescriptors = pca.project(frameDescriptors); //queryDescriptors = PcaDecColumn(queryDescriptors, 64); //frameDescriptors = PcaDecColumn(frameDescriptors, 64);

It work, but match size reduce from 130 to 20, PCA can speed up match but also reduce match size, how to improve it.

tczengming gravatar imagetczengming ( 2018-08-02 21:05:06 -0600 )edit

imho, you're on the wrong bus here, with both PCA and feature matching.

i think, you need something like this instead, with an SVM(if you want to do classification) or a flann::Index (if the goal is clustering / nearest neighbour retrieval)

berak gravatar imageberak ( 2018-08-03 01:01:12 -0600 )edit

My goal is match one page from a picture book, i have try use BOW without PCA, BOW is fast but accuracy not good enough, i only need one target, BOW may find several similar.

tczengming gravatar imagetczengming ( 2018-08-03 02:06:29 -0600 )edit