Reeven's profile - activity

In Xcode I made a static library (a .a file) implementing the header and source for my Load and Predict methods. That static library was linked to the OpenCV for iOS framework. I had to make sure to copy the framework's Headers and Resources subfolders and opencv2 file, all located in Versions\A, back to the root of the framework.

I could then place the static library and the OpenCV for iOS framework in my Unity iOS plugin folder, and use the library from Unity script as I would a .dll or .so through [DllImport ("__Internal")].

From my understanding, basic iOS C++ plugins in Unity are dead simple. Simply jot down working code in a .mm text file and shove that file into Assets/Plugin/iOS. Even simpler than making a windows .dll plugin.

But... what about an iOS plugin that itself makes calls to another framework like, say, OpenCV for iOS? How do you include that? The documentation beyond the most basic examples like the one I linked is sparse. Do I just dump opencv2.framework into Assets/Plugin/iOS and cross my fingers my own C++ code will know to talk to OpenCV's through some kind of magic? I am pretty sure there must be some additional steps, probably once the generated project's gotten to Xcode's side of things. What would these steps be?

Also, includes and such: how would these work in the .mm file?

Any help or link to a tutorial or documentation pertinent to my use case highly appreciated. Thanks!

EDIT: As a point of comparison, here is the C++ .cpp code for my Editor / Windows version of things, its methods being called by Unity through [DllImport]. For Editor / Windows, this works perfectly :

#include "MEHSvm.h"

extern "C"
{
    // create global bow vocabulary
    cv::Ptr<cv::DescriptorMatcher> matcher = cv::DescriptorMatcher::create("BruteForce-Hamming");
    cv::Ptr<cv::DescriptorExtractor> extractor = cv::ORB::create();
    cv::Ptr<cv::FeatureDetector> detector = cv::ORB::create();
    cv::BOWImgDescriptorExtractor bowDE(extractor, matcher);

    cv::Ptr<cv::ml::SVM> svm = cv::ml::SVM::create();

    int Load(const char* svmPath, const char* vocabularyPath)
    {
        svm = svm->load(svmPath);

        cv::Mat vocabulary;
        cv::FileStorage storage(vocabularyPath, cv::FileStorage::READ);
        storage.getFirstTopLevelNode() >> vocabulary;
        storage.release();
        bowDE.setVocabulary(vocabulary);

        return svm->getType(); //should be 100 for C_SVC
    }

    int Predict(uchar* bytes, int width, int height, int type)
    {
        cv::Mat image = cv::Mat(height, width, type, bytes);

        std::vector<cv::KeyPoint> keypoints;
        detector->detect(image, keypoints);
        cv::Mat descriptors;
        bowDE.compute(image, keypoints, descriptors);
        if (descriptors.empty()) return -1;

        return svm->predict(descriptors);
    }
}

Maybe something like this? https://zami0xzami.wordpress.com/2016...

Oh, I want to use JNI / NDK. I know that's the part I need to understand to have working C++ in Android. But it is also the part I'm absolutely not familiar with. It's OpenCV Java and OpenCV4Android that are a no go, JNI / NDK are fine.

Context: I'm using OpenCV for a Unity project. I already made a simple .dll plugin to wrap the C++ code I use to fit my needs on Editor / PC. Now I need a .so or .jar plugin to do the same for Android compatibility. For the life of me, I don't understand all the JNI / NDK stuff. I must not use OpenCV Java or OpenCV4Android, as these two have some bugs / missing features that I'm actually using: in particular, they're missing BOWs and the ability to load SVMs.

All current tutorials I have googled either fall back to OpenCV4Android or assume familiarity with the JNI / NDK, familiarity which I have zero of.

Question: How do I include and use C++ OpenCV in Android Studio? Explain it to me like I'm five.

EDIT: Still stuck.

get builded Libraries

Which builded libraries? Built how? Found where? Certainly not the .lib or .dll from my Windows version? Can I just dump the OpenCV headers and class files into the project's cpp folder and go from there?

So, how did this turn out? It's been three years and BOWImgDescriptorExtractor and BOWTrainer are still out.

Thank you, this is exactly what I needed.

Hello!

Context: I am re-implementing the kmeans algorithm into a kmajority algorithm for binary descriptors, like ORB, based on hamming distances, inspired by this paper. This requires, for every binary descriptor in a cluster, to make each bit position of each descriptor "vote" for either 0 or 1 for the final value of that position in the cluster center.

Descriptor 1: 0 0 0 0 1 0 1 0 ...
Descriptor 2: 1 0 1 0 1 0 1 0 ...
Descriptor 3: 1 0 0 0 1 0 0 0 ...
---------------------------------
Center      : 1 0 0 0 1 0 1 0 ... <-- after majority vote for each bit

Question: ORB descriptors are stored in their mats as uchar, not bits. They are stored in a Mat of N rows (1 row per descriptor) and 32 columns (32 uchar for 256 bits). Obviously, I'm not interested in uchars, it's the bits I want. Therefore, is requesting the rows with:

bool* descriptor = myMat .ptr<bool>(rowIndex)

... the right approach if I want to iterate over a descriptor bit by bit instead of uchar by uchar? (I'm pretty unfamiliar with C++).

Context: As you might be aware, the current OpenCV BOW implementation can only function as expected with float descriptors (like SIFT and SURF) since the current clustering method is based on KMeans of euclidean distances with L2 norms. On the other hand, the much faster binary descriptors (like ORB or BRIEF) favored in mobile and real-time contexts cannot be used with the current BOW implementation, as any notion of "distance" between binary descriptors would have to be based on Hamming distance and some form of voting scheme to determine cluster centroids instead. Such a cluster method does not currently exist within the versions of OpenCV I know, 2.4 and 3.1.

The sometimes (wrongly) suggested solution of converting binary descriptors from CV_8U to CV_32F to plug them into the current BOW cluster medhod, though attractive as a quick-and-dirty fix, is nothing more than trying to fit a square peg into a round slot: it makes no mathematical sense, it's garbage in, garbage out.

Relevant links:

http://answers.opencv.org/question/17... http://answers.opencv.org/question/24... http://stackoverflow.com/questions/28... http://imagelab.ing.unimore.it/imagel... https://github.com/opencv/opencv/issu...

Question: Is a BOW implementation compatible with binary descriptors being worked on? If there is not currently one in some experimental / beta version of the latest OpenCV, is there any code example or tutorial of how to extend / inherit / modify the current BOW to make a compatible version for myself? The links above discuss the logic and algorithm of such a theoretical binary BOW, but otherwise I am basically a C++ and OpenCV newbie; I wouldn't know even where to begin to implement such a new BOW myself.

Answer: I was loading the SVM incorrectly, as per this other question:

http://answers.opencv.org/question/93...

in opencv3, Algorithm::load() creates a new instance, so you have to load your SVM like:

Ptr<ml::svm> classifier = Algorithm::load<ml::svm>(filename); (in your original code, the newly loaded model was discarded, never transferred to your classifier instance)