How to use bag of words example with BRIEF descriptors?

edit

There is a terrible mistake. A simple conversion of a uchar to float is NOT the right solution. If you look at the details of KMeans, one must minimize the sum of distances :

• for a real-valued vector, this corresponds to a mean

• for a binary vector stored in uchar, this does NOT correspond to a mean

Indeed for each coordinate inside the uchar one must make a vote between the ones and the zeros to get a medoid.

Here is the relevant part of the code to compute BoW with ORB:

for( all your images )
{
    // Detect interesting points
    orb(img, Mat(), keypoints, descriptors);
    // Keep characteristics of images for further clustering.
    characteristics.push_back(descriptors);
}
// Create the BOW object with K classes
BOWKMeansTrainer bow(K);
for( all your descriptors )
{
    // Convert characteristics vector to float
    // This is require by BOWTrainer class
    Mat descr;
    characteristics[k].convertTo(descr, CV_32F);
    // Add it to the BOW object
    bow.add(descr);
}
// Perform the clustering of stored vectors
Mat voc = bow.cluster();

You need to convert features from CV_8U to CV_32F, because bag of words object expects float descriptors. It is not required for SIFT or SURF descriptors because they are in float, as said by @berak.

If you have some issues with this code, please show us some relevant parts of yours.

Hi,

I had same problem and when I change type of dictionary float to unsigned char it works.

cv::Mat floatdictonary; // floatDictionary = bowTrainer.cluster(floatFeaturesUnclustered);//you have convert features //unsigned char to float cv::Mat udictionary; floatdictionary.convertTo(udictonary,CV_8UC1); // bowDE->setVocabulary(udictonary); Mat bowDescriptor; std::vector<std::vector<int> > pointIdxOfClusters; bowDE.compute(img, keypoints, bowDescriptor,&pointIdxOfClusters);

The vocabulary that is computed using the cluster() function in BOWKMeansTrainer is of type CV_32FC1. The descriptors of the image against which to match the vocabulary is of type CV_8UC1, when computed using non-sift and non-surf descriptor extractors. One gets this error ""OpenCV Error: Assertion failed (queryDescriptors.type() == trainDescCollection[0].type()) in knnMatchImpl, file /Users/jamescharters/Downloads/opencv-2.4.6.1/modules/features2d/src/matchers.cpp, line 351"" when one tries to match descriptors of different types .

To solve all the problems just combine the answers of Mathieu Barnachon and igsgk:

//first convert uchar to float
cv::Mat descriptors;
extractorIn->compute(img, keypoints, descriptors);
cv::Mat descriptorsFloat;
descriptors.convertTo(descriptorsFloat, CV_32FC1);
bowTrainerInOut.add(descriptorsFloat);

// ... and then set the vocabulary as uchar after clustering
cv::Mat dictionary = bowTrainer.cluster();
cv::Mat uDictionary;
dictionary.convertTo(uDictionary, CV_8UC1);
bowDE.setVocabulary(uDictionary);

This solved it for my case

Just reposting from the comments above with nice code formatting.

In short, there appears to be an issue when using the BOWImgDescriptorExtractor on the above ORB BoW approach. This might trip up other people, and seems closely related to the above question.

Here is some sample code:

int dictionarySize = 1024;
TermCriteria tc(CV_TERMCRIT_ITER, 10, 0.00001);
int retries = 1;
int flags = KMEANS_PP_CENTERS;

// How the objects are set up
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("BruteForce");
Ptr<DescriptorExtractor> extractor = DescriptorExtractor::create("ORB");
Ptr<FeatureDetector> detector = FeatureDetector::create("Dense");
BOWKMeansTrainer bowTrainer(dictionarySize, tc, retries, flags);
BOWImgDescriptorExtractor bowDE(extractor, matcher)

// ...
// ... code here is equivalent to answer above for computing BoW but has been excluded for brevity ... 
// ...

// Read in the image that we want to match against the descriptor
Mat img = imread(entryPath.string(), 1);
cvtColor(img, img, CV_BGR2GRAY);

// Get the keypoints
vector<KeyPoint> keypoints;
detector->detect(img, keypoints);

// Try to get the descriptor
Mat bowDescriptor = Mat(0, 32, CV_32F);
bowDE.compute(img, keypoints, bowDescriptor); // <-- errors here

And the error is:

"OpenCV Error: Assertion failed (queryDescriptors.type() == trainDescCollection[0].type()) in knnMatchImpl, file /Users/jamescharters/Downloads/opencv-2.4.6.1/modules/features2d/src/matchers.cpp, line 351"

I am doing this in C++ using OpenCV 2.4.6.1 on Mac OS 10.8.4.