testflyjets's profile - activity

I did another experiment on Android. I wrote a native method using JNI to call the FeatureDetector.detect() method and examined the keypoints generated. They are the same keypoints generated by the Android Java method, so at least those two are consistent.

I did a small experiment and took the descriptor matrix generated on Android and shuffled all the rows randomly, then used that matrix to try to register the target image. It doesn't work, just as the descriptor matrix generated in C++ has a different row ordering and also doesn't work.

So it seems the order of rows in the descriptor matrix does matter in terms of calling .match().

I also noted that there is a 1:1 correspondence between the rows in the keypoints and the generated descriptor matrix rows. That is, for each keypoint generated on Android, the same feature descriptor row is generated on Android and C++. So if the keypoints are in a different order in C++, the descriptor matrix rows will be in the same, different order.

Thanks @StevenPuttemans for the information. The exact same image, in OpenCV Mat format, is passed to the FeatureDetector in both C++ and Android, and the exact same set of keypoints and descriptors is generated.

We are using the BRUTEFORCE_HAMMING DescriptorMatcher, so based on what you said it would seem that even with the descriptors in a different order the matches should be made between the training image and the test image.

@berak, if you have any suggestions on the Android side for debugging this I'd be very appreciative.

I'm trying to understand differences I'm seeing between the C++ and Android implementations of FeatureDetector and DescriptorExtractor in OpenCV 2.4.9.

Fundamentally I'm trying to generate a descriptor matrix for an image in C++, encode it in JSON, pass it to an Android app, and use that descriptor matrix in the DescriptorMatcher.match() method for image recognition.

I'm using ORB for both feature detection and descriptor extraction in both C++ and Android.

What I'm finding is that the C++ descriptor extractor generates the same descriptor matrix as the Android implementation but with the order of the rows completely scrambled. When decoding this matrix from JSON in Android the match() method doesn't find matches.

I also looked at the keypoints generated by the FeatureDetector on both sides. Again, the actual values of the rows in the generated Mat structure are the same, but the rows are in different order. And this order seems to drive the DescriptorExtractor to generate a different order for the descriptor matrix rows.

First question: does the order of the rows in a descriptor matrix fundamentally matter to the DescriptorMatcher.match() method? It appears it does, but I can't find any documentation that says it should.

Second question: does the order of keypoints from the FeatureDetector matter to the DescriptorExtractor.compute() method? Again, empirically, it seems so but I can't find any documentation that states it explicitly.

Final question: Why would the C++ feature detector return keypoints in a different order for the exact same image? I've verified that the image, in Mat structure, is exactly the same in both implementations.

In both C++ and Android I'm using the default create() methods for the detectors and extractors so I would expect the objects to be initialized the same way. Is this a faulty assumption? I can't seem to directly inspect the detector or extractor in Android to verify this.

If I take the descriptor matrix generated in Android and encode it into JSON (using Base64 encoding for the data element), and pass that into the Android app instead of the JSON-encoded descriptor matrix created by C++, the descriptor matcher works. This tells me the issue isn't with the JSON or Base64 encoding.

If anyone has suggestions on where to look next I'd be much obliged.

Thanks,

Chris