Ask Your Question

Reeven's profile - activity

2018-08-06 00:12:00 -0500 received badge  Notable Question (source)
2017-08-06 05:22:11 -0500 received badge  Popular Question (source)
2016-11-03 08:51:02 -0500 answered a question Unity iOS C++ plugin that itself relies on OpenCV

In Xcode I made a static library (a .a file) implementing the header and source for my Load and Predict methods. That static library was linked to the OpenCV for iOS framework. I had to make sure to copy the framework's Headers and Resources subfolders and opencv2 file, all located in Versions\A, back to the root of the framework.

I could then place the static library and the OpenCV for iOS framework in my Unity iOS plugin folder, and use the library from Unity script as I would a .dll or .so through [DllImport ("__Internal")].

2016-10-28 08:05:09 -0500 asked a question Unity iOS C++ plugin that itself relies on OpenCV

From my understanding, basic iOS C++ plugins in Unity are dead simple. Simply jot down working code in a .mm text file and shove that file into Assets/Plugin/iOS. Even simpler than making a windows .dll plugin.

But... what about an iOS plugin that itself makes calls to another framework like, say, OpenCV for iOS? How do you include that? The documentation beyond the most basic examples like the one I linked is sparse. Do I just dump opencv2.framework into Assets/Plugin/iOS and cross my fingers my own C++ code will know to talk to OpenCV's through some kind of magic? I am pretty sure there must be some additional steps, probably once the generated project's gotten to Xcode's side of things. What would these steps be?

Also, includes and such: how would these work in the .mm file?

Any help or link to a tutorial or documentation pertinent to my use case highly appreciated. Thanks!

EDIT: As a point of comparison, here is the C++ .cpp code for my Editor / Windows version of things, its methods being called by Unity through [DllImport]. For Editor / Windows, this works perfectly :

#include "MEHSvm.h"

extern "C"
    // create global bow vocabulary
    cv::Ptr<cv::DescriptorMatcher> matcher = cv::DescriptorMatcher::create("BruteForce-Hamming");
    cv::Ptr<cv::DescriptorExtractor> extractor = cv::ORB::create();
    cv::Ptr<cv::FeatureDetector> detector = cv::ORB::create();
    cv::BOWImgDescriptorExtractor bowDE(extractor, matcher);

    cv::Ptr<cv::ml::SVM> svm = cv::ml::SVM::create();

    int Load(const char* svmPath, const char* vocabularyPath)
        svm = svm->load(svmPath);

        cv::Mat vocabulary;
        cv::FileStorage storage(vocabularyPath, cv::FileStorage::READ);
        storage.getFirstTopLevelNode() >> vocabulary;

        return svm->getType(); //should be 100 for C_SVC

    int Predict(uchar* bytes, int width, int height, int type)
        cv::Mat image = cv::Mat(height, width, type, bytes);

        std::vector<cv::KeyPoint> keypoints;
        detector->detect(image, keypoints);
        cv::Mat descriptors;
        bowDE.compute(image, keypoints, descriptors);
        if (descriptors.empty()) return -1;

        return svm->predict(descriptors);
2016-09-30 10:15:24 -0500 commented answer OpenCV C++ in Android Studio

Maybe something like this?

2016-09-30 07:16:26 -0500 commented question OpenCV C++ in Android Studio

Oh, I want to use JNI / NDK. I know that's the part I need to understand to have working C++ in Android. But it is also the part I'm absolutely not familiar with. It's OpenCV Java and OpenCV4Android that are a no go, JNI / NDK are fine.

2016-09-29 15:57:05 -0500 asked a question OpenCV C++ in Android Studio

Context: I'm using OpenCV for a Unity project. I already made a simple .dll plugin to wrap the C++ code I use to fit my needs on Editor / PC. Now I need a .so or .jar plugin to do the same for Android compatibility. For the life of me, I don't understand all the JNI / NDK stuff. I must not use OpenCV Java or OpenCV4Android, as these two have some bugs / missing features that I'm actually using: in particular, they're missing BOWs and the ability to load SVMs.

All current tutorials I have googled either fall back to OpenCV4Android or assume familiarity with the JNI / NDK, familiarity which I have zero of.

Question: How do I include and use C++ OpenCV in Android Studio? Explain it to me like I'm five.

EDIT: Still stuck.

get builded Libraries

Which builded libraries? Built how? Found where? Certainly not the .lib or .dll from my Windows version? Can I just dump the OpenCV headers and class files into the project's cpp folder and go from there?

2016-09-22 10:40:12 -0500 commented question BOWTrainer in java api

So, how did this turn out? It's been three years and BOWImgDescriptorExtractor and BOWTrainer are still out.

2016-09-21 15:29:53 -0500 commented answer Getting ORB descriptor values bit by bit

Thank you, this is exactly what I needed.

2016-09-21 07:38:20 -0500 asked a question Getting ORB descriptor values bit by bit


Context: I am re-implementing the kmeans algorithm into a kmajority algorithm for binary descriptors, like ORB, based on hamming distances, inspired by this paper. This requires, for every binary descriptor in a cluster, to make each bit position of each descriptor "vote" for either 0 or 1 for the final value of that position in the cluster center.

Descriptor 1: 0 0 0 0 1 0 1 0 ...
Descriptor 2: 1 0 1 0 1 0 1 0 ...
Descriptor 3: 1 0 0 0 1 0 0 0 ...
Center      : 1 0 0 0 1 0 1 0 ... <-- after majority vote for each bit

Question: ORB descriptors are stored in their mats as uchar, not bits. They are stored in a Mat of N rows (1 row per descriptor) and 32 columns (32 uchar for 256 bits). Obviously, I'm not interested in uchars, it's the bits I want. Therefore, is requesting the rows with:

bool* descriptor = myMat .ptr<bool>(rowIndex)

... the right approach if I want to iterate over a descriptor bit by bit instead of uchar by uchar? (I'm pretty unfamiliar with C++).

2016-09-20 13:53:24 -0500 received badge  Enthusiast
2016-09-14 15:10:55 -0500 asked a question Bag of Word BOWTrainer clustering for binary descriptors

Context: As you might be aware, the current OpenCV BOW implementation can only function as expected with float descriptors (like SIFT and SURF) since the current clustering method is based on KMeans of euclidean distances with L2 norms. On the other hand, the much faster binary descriptors (like ORB or BRIEF) favored in mobile and real-time contexts cannot be used with the current BOW implementation, as any notion of "distance" between binary descriptors would have to be based on Hamming distance and some form of voting scheme to determine cluster centroids instead. Such a cluster method does not currently exist within the versions of OpenCV I know, 2.4 and 3.1.

The sometimes (wrongly) suggested solution of converting binary descriptors from CV_8U to CV_32F to plug them into the current BOW cluster medhod, though attractive as a quick-and-dirty fix, is nothing more than trying to fit a square peg into a round slot: it makes no mathematical sense, it's garbage in, garbage out.

Relevant links:

Question: Is a BOW implementation compatible with binary descriptors being worked on? If there is not currently one in some experimental / beta version of the latest OpenCV, is there any code example or tutorial of how to extend / inherit / modify the current BOW to make a compatible version for myself? The links above discuss the logic and algorithm of such a theoretical binary BOW, but otherwise I am basically a C++ and OpenCV newbie; I wouldn't know even where to begin to implement such a new BOW myself.

2016-09-13 10:11:34 -0500 received badge  Supporter (source)
2016-09-13 06:59:14 -0500 received badge  Teacher (source)
2016-09-12 11:16:28 -0500 received badge  Self-Learner (source)
2016-09-12 08:56:20 -0500 received badge  Scholar (source)
2016-09-12 08:56:00 -0500 answered a question Training SVM for image recognition with BOW: error on prediction

Answer: I was loading the SVM incorrectly, as per this other question:

in opencv3, Algorithm::load() creates a new instance, so you have to load your SVM like:

Ptr<ml::svm> classifier = Algorithm::load<ml::svm>(filename); (in your original code, the newly loaded model was discarded, never transferred to your classifier instance)

2016-09-12 08:38:19 -0500 received badge  Editor (source)
2016-09-09 21:52:23 -0500 asked a question Training SVM for image recognition with BOW: error on prediction


Context: I am currently attempting to train a SVM to recognize a specific building in a scene vs. different buildings. After attempting simpler approaches like direct image feature matching + homography and such, I want to attempt something a bit more flexible and/or powerful. Hence, I have set my gaze upon BOW+SVM.

Problem: As far as I am aware, the training phase goes without a hitch. However, when I reuse (either straight in code or after reloading the saved text format) my trained vocabulary and SVM to attempt a prediction on a new image, I get an out of memory error on the svm.predict(descriptors) function:

Unhandled exception at 0x7594C41F in MachineLearningTrainerBOW.exe: Microsoft C++ exception: cv::Exception at memory location 0x0037F288.

I honestly don't know what's wrong, as I have followed examples setup in a similar manner to my current code and at a glance I do not seem to do anything outlandish or different than what they do:

Could anyone point the way to a complete OpenCV tutorial or exemple covering the use of BOW+SVM, or maybe point out what I might be doing wrong from my code? Thank you!

// cluster count
const int CLUSTER_COUNT = 1000;

// create global bow vocabulary with TermCriteria as per tutorial
Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");
Ptr<DescriptorExtractor> extractor = SurfDescriptorExtractor::create();
BOWImgDescriptorExtractor bowDE(extractor, matcher);
Ptr<SURF> detector = SURF::create(400);

//... some irrelevant functions for loading files and such...

void ComputeBow(vector<Mat> positives, vector<Mat> negatives, string vocabularySave)
    // get SURF descriptors and add to BOW each input files
    cout << "Acquiring descriptors, this might take a while... ";
    for (int i = 0; i < positives.size(); i++)
        vector<KeyPoint> keypoints;
        detector->detect(positives[i], keypoints);
        Mat descriptors;
        extractor->compute(positives[i], keypoints, descriptors);
        if (!descriptors.empty()) bowTrainer.add(descriptors);

    /*for (int i = 0; i < negatives.size(); i++)
        vector<KeyPoint> keypoints;
        detector->detect(negatives[i], keypoints);
        Mat descriptors;
        extractor->compute(negatives[i], keypoints, descriptors);
        if (!descriptors.empty()) bowTrainer.add(descriptors);
    cout << "Description complete!" << endl;

    // Create the vocabulary with KMeans.
    cout << "Clustering features, this might take a while... ";
    Mat vocabulary;
    vocabulary = bowTrainer.cluster();
    SaveVocabulary(vocabulary, vocabularySave);

    cout << "Clustering complete!" << endl;

void TrainSVM(vector<Mat> positiveMats, vector<Mat> negativeMats, string svmSave, string vocabularySave)
    //Setup the BOW
    ComputeBow(positiveMats, negativeMats, vocabularySave);

    // create training data positive and negative
    Mat train, response;
    cout << "Creating training sets, this might take a while... ";
    for (int i = 0; i < positiveMats.size(); i++)
        // set training data using BOWImgDescriptorExtractor
        vector<KeyPoint> keypoints;
        detector->detect(positiveMats[i], keypoints);
        Mat descriptors;
        bowDE.compute(positiveMats[i], keypoints, descriptors);
        if (!descriptors.empty())
            train.push_back(descriptors);     // update training data
            response.push_back(1);        // update response data

    for (int i = 0; i < negativeMats.size(); i++)
        // set training data using BOWImgDescriptorExtractor
        vector<KeyPoint> keypoints;
        detector->detect(negativeMats[i], keypoints);
        Mat descriptors ...