Revision history [back]

Hello,

I am trying to use flann to create a database of large set of descriptors and later querying it for image search. I find it really weird that I have to create a super mat (all descriptor mat's concatenated using vconcat. The problems are many:

1. There is no way of mapping matched descriptors back to original image, but Lowe's Flann implemented (original one) does allow this. There should be a constructor which accept vector<mat> during index creation. Result indices and distances should contain imgIdx, to map it back to original image.
2. Storing index to disk seem to work fine but when I am loading it back, it expects me to know the exact size of the dataset which was stored. How do I suppose to know that. The purpose of storing in file itself is defeated. The loading part should not expect any Size of Mat as input.
3. When I am storing indices to file, the size of Mat is (WxH) but when I am loading it, it expects me to supply transposed size which is (HxW). This is weird.
4. Even after doing all this, I am hitting segfaults.

Anyone who has worked on opencv flann, storing and loading from disk, with multiple images, and mapping descriptors back to original image ?

Sample codes:

void computeAndSaveIndices(string directory, string outputFile) {
    vector<string> images;
    vector<Mat> descriptors;
    getFilesofExtension(directory, images, ".key");

    for (vector<string>::iterator it1 = images.begin() ; it1 != images.end(); ++it1)
    {
        cout << "Loading descriptors from file: " << *it1 << "\n";
        string file = *it1;
        Mat mat = Mat();
        readMatFromFile(file, mat);
        if (mat.rows > 0) {
            descriptors.push_back(mat);
        }
    }

    cout << "creating index using " << descriptors.size() << " files \n";

    **Mat superMat;
    vconcat(descriptors, superMat);

    cout << "supermat size is  [" << superMat.rows  <<  " " << superMat.cols << "] \n";**

    flann::LshIndexParams lshIndexParams(20,10,2);

    flann::Index kdtree(superMat, lshIndexParams, cvflann::FLANN_DIST_HAMMING);


    cout << "outputting index file to disk: " << "\n";

    kdtree.save(outputFile);

}

float searchInSavedIndices(string indicesFile, string descriptorFile) {

    **Mat indexMat = Mat(Size(61, 76728), CV_8UC1);**

    // read saved indices
    flann::Index kdtree(indexMat,
                        flann::SavedIndexParams(indicesFile),
                        cvflann::FLANN_DIST_HAMMING);


    // read query descriptors from file
    Mat descriptors = Mat();
    readMatFromFile(descriptorFile, descriptors);

    // prepare to output indices and distances
    Mat indices = Mat(descriptors.rows, 2, CV_32S);
    Mat dists = Mat(descriptors.rows, 2, CV_32S);

    cout << "Searching " << descriptorFile << " in " << indicesFile << "\n";
    kdtree.knnSearch(descriptors, indices, dists, 2);

    // indices should contain image idx too.

    int goodMatchCount = 0;

    for (int i=0; i<dists.rows; i++)
    {
        if (dists.at<int>(i, 1) > 0 && dists.at<int>(i,0) < 50)
        {
            goodMatchCount++;
        }

    }

    float percentage = (goodMatchCount* 100.f)/descriptors.rows;
    return percentage;
}