Ask Your Question

Revision history [back]

Keypoint and descriptor ordering - C++ vs Android

I'm trying to understand differences I'm seeing between the C++ and Android implementations of FeatureDetector and DescriptorExtractor in OpenCV 2.4.9.

Fundamentally I'm trying to generate a descriptor matrix for an image in C++, encode it in JSON, pass it to an Android app, and use that descriptor matrix in the DescriptorMatcher.match() method for image recognition.

I'm using ORB for both feature detection and descriptor extraction in both C++ and Android.

What I'm finding is that the C++ descriptor extractor generates the same descriptor matrix as the Android implementation but with the order of the rows completely scrambled. When decoding this matrix from JSON in Android the match() method doesn't find matches.

I also looked at the keypoints generated by the FeatureDetector on both sides. Again, the actual values of the rows in the generated Mat structure are the same, but the rows are in different order. And this order seems to drive the DescriptorExtractor to generate a different order for the descriptor matrix rows.

First question: does the order of the rows in a descriptor matrix fundamentally matter to the DescriptorMatcher.match() method? It appears it does, but I can't find any documentation that says it should.

Second question: does the order of keypoints from the FeatureDetector matter to the DescriptorExtractor.compute() method? Again, empirically, it seems so but I can't find any documentation that states it explicitly.

Final question: Why would the C++ feature detector return keypoints in a different order for the exact same image? I've verified that the image, in Mat structure, is exactly the same in both implementations.

In both C++ and Android I'm using the default create() methods for the detectors and extractors so I would expect the objects to be initialized the same way. Is this a faulty assumption? I can't seem to directly inspect the detector or extractor in Android to verify this.

If I take the descriptor matrix generated in Android and encode it into JSON (using Base64 encoding for the data element), and pass that into the Android app instead of the JSON-encoded descriptor matrix created by C++, the descriptor matcher works. This tells me the issue isn't with the JSON or Base64 encoding.

If anyone has suggestions on where to look next I'd be much obliged.