Ask Your Question
1

Why does cv::DescriptorExtractor::compute return empty descriptors for a std::vector<cv::Mat>?

asked 2014-10-02 04:44:20 -0500

Jan Marek gravatar image

First, I already asked this question on Stackoverflow but I did not find a solution so far or it might be a bug. Therefor I repeat my question here: The problem is the behaviour of feature extractors in openCV if one wants to extract descriptors of several images at the same time. The cv::DescriptorExtractor offers two interfaces for that:

void DescriptorExtractor::compute(const Mat& image, vector<KeyPoint>& keypoints, Mat& descriptors) const
void DescriptorExtractor::compute(const vector<Mat>& images, vector<vector<KeyPoint>>& keypoints, vector<Mat>& descriptors) const

I am talking about the second signature which allows to pass a sequence of images. I tested the behaviour with the following minimal example:

#include"opencv2\opencv.hpp"
#include "opencv2\xfeatures2d\nonfree.hpp"

int main(){

    //Read image
    cv::Mat img = cv::imread("./sample.png");
    //Create vector of images
    std::vector<cv::Mat> imageSequence(10, img);
    //Create SIFT detector and extractor
    cv::FeatureDetector* detector = new cv::xfeatures2d::SiftFeatureDetector();
    cv::DescriptorExtractor* extractor = new cv::xfeatures2d::SiftDescriptorExtractor();
    //Keypoints
    std::vector<std::vector<cv::KeyPoint>> keypoints;
    //Compute keypoints
    detector->detect(imageSequence, keypoints);
    //This vector should contain all descriptors
    std::vector<cv::Mat> allDescriptors;
    //Version 1: Using a for loop and the single image signature of DescriptorExtractor::compute works.
    for (int i = 0; i < imageSequence.size(); ++i){
        cv::Mat descriptors;
        //This compute(...) works
        extractor->compute(imageSequence.at(i), keypoints.at(i), descriptors);
        allDescriptors.push_back(descriptors);
    }

    allDescriptors.clear();
    //Version 2: Using the interface for the computation of descriptors for std::vector<cv::Mat> fails
    extractor->compute(imageSequence, keypoints, allDescriptors);

    return 0;
}

While Version 1 in the example above returns the descriptors for all images correctly, the Version 2 does not - the variable allDescriptors is empty. However, when I step into the compute function I can observe that it calls the compute function with the single image signature in a for loop but the original variable allDescriptors is unaffected. I am working with the latest 3.0 version of opencv from the git repository. It seems that the computed descriptors are computed but not stored in allDescriptors and the result is lost after return.

edit retag flag offensive close merge delete

Comments

1

Be careful, you do not delete the pointers... Better use cv::Ptr<>;

thdrksdfthmn gravatar imagethdrksdfthmn ( 2014-10-02 05:03:05 -0500 )edit
1

this looks like a bug. std::vector<Mat> descCollection gets destroyed after leaving DescriptorExtractor::compute( InputArrayOfArrays _imageCollection,...)

berak gravatar imageberak ( 2014-10-02 08:27:17 -0500 )edit

@berak: Thanks for testing and confirming the behaviour. @thdrksdfthmn: Thanks for pointing out. In my project I use QSharedPoitners but I changed to simple pointers for the minimal working example and forgot to add the delete.

Jan Marek gravatar imageJan Marek ( 2014-10-06 04:39:20 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2014-10-02 08:55:45 -0500

berak gravatar image

updated 2014-10-02 10:00:32 -0500

not an answer, but it's easily reproducable in 3.0 (local is unfortunately, what it says ):

void my(OutputArrayOfArrays oaa)
{
    std::vector<cv::Mat> local;
    oaa.getMatVector(local);
    local.resize(4);
    cerr << "local: " << local.size() << endl;;
}


int main()
{   
    std::vector<cv::Mat> vm;
    my(vm);
    cerr << "vm:    " << vm.size() << endl;;
    return 2;
}

local: 4
vm:    0
edit flag offensive delete link more

Comments

actually, trying this on 2.4.9 gives same result

berak gravatar imageberak ( 2014-10-06 04:54:49 -0500 )edit

@Jan Marek, do you want to make an an issue here, or should i ?

berak gravatar imageberak ( 2014-10-06 04:58:57 -0500 )edit

ok, question answered ;)

berak gravatar imageberak ( 2014-10-06 06:49:19 -0500 )edit
1

answered 2014-10-02 05:07:19 -0500

thdrksdfthmn gravatar image

I have changed your code a little, using cv::Ptr instead of pointers:

cv::Ptr< cv::FeatureDetector > detector = new cv::SiftFeatureDetector();
cv::Ptr< cv::DescriptorExtractor > extractor = new cv::SiftDescriptorExtractor();

And using OpenCV 2.4.9 it works file, the allDescriptors is not empty at the end.

By the way, I have not found "opencv2\xfeatures2d\nonfree.hpp", so I have used #include <opencv2\nonfree\features2d.hpp>. Maybe it is a problem of 3.0.0?

edit flag offensive delete link more

Comments

1

^^ SURF and SIFT have been moved to the opencv_contrib repo in 3.0, to xfeatures2d, so the includes look different

berak gravatar imageberak ( 2014-10-02 07:24:25 -0500 )edit

Then you can test the code and give an answer ;)

thdrksdfthmn gravatar imagethdrksdfthmn ( 2014-10-02 07:26:43 -0500 )edit

Thanks for testing the code with 2.4.9. As berak pointed out, the compute version of 3.0 does not work correctly because it stores the result in a local variable. This seems to be a bug in 3.0.

Jan Marek gravatar imageJan Marek ( 2014-10-06 04:48:53 -0500 )edit
Login/Signup to Answer

Question Tools

Stats

Asked: 2014-10-02 04:44:20 -0500

Seen: 1,537 times

Last updated: Oct 02 '14