JesseCh's profile - activity

I found the answer and case closed. I use a Matlab tool box: GML Adaboost solve this problem. I transferred my csv files into .mat, and the dataset is compressed into a 2GB .Mat file from a 60 GB .csv. The model can be trained by this toolbox and export as a txt file. This model can be used in my C++ program by using a parser which provided by GML Adaboost .

Hi all, I am trying to train my dataset (over 15G). Obviously, it is not possible to load entire data set into memory at once. Therefore, I am considering to load my data separately, and it worked fine on my own implementation of adaboost.

Now, I would like to train this dataset by OpenCV. I found the training data is store into a smart pointer

Ptr<ml::TrainData>

I need to load all my data, because training process on OpenCV can only input one set of TrainData according to the following OpenCV 3 source code boost.cpp:

bool train( const Ptr<TrainData>& trainData, int flags )
{
    startTraining(trainData, flags);
    int treeidx, ntrees = bparams.weakCount >= 0 ? bparams.weakCount : 10000;
    vector<int> sidx = w->sidx;

    for( treeidx = 0; treeidx < ntrees; treeidx++ )
    {
        int root = addTree( sidx );
        if( root < 0 )
            return false;
        updateWeightsAndTrim( treeidx, sidx );
    }
    endTraining();
    return true;
}

Is there any other function can make me divide my dataset into several chunks and put into training process?

If not, does anyone know other adaboost library can handle huge size of data?

I had tried both .ptr<float> and .ptr<int> and still got exact same result.

Hi all, I am trying to train my own boosting model, but I encountered Assertion failed on trainning stage. My program is trying to read a CSV file into cv::Mat and use cv::Mat to be the input of trainning process. Following is my code:

int main(int argc, char *argv[]){

int rows = 10;
int cols = 4;
string pixel;

Mat img(Size(cols,rows),CV_32F);
Mat response(Size(1,rows),CV_32F);

ifstream file("D:/testFile2/test.csv", ifstream::in);
for(int i=0; i<rows; i++){
    float* data = (float*)img.ptr<ushort>(i);
    float* data2 = (float*)response.ptr<ushort>(i);

        for(int j=0; j<cols+1; j++){

            if(j==0){
                getline(file, pixel, ',');
                data2[j] = (float)atof(pixel.c_str());

            }
            else if(j == cols){
                getline(file, pixel, '\n');
                data[j-1] = (float)atof(pixel.c_str());

            }
            else{
                getline(file, pixel, ',');
                data[j-1] = (float)atof(pixel.c_str());

            }

        }

}

 printf("Data Read\n");

 Ptr<ml::TrainData> dataset = ml::TrainData::create(img,ml::SampleTypes::ROW_SAMPLE,response);
 Ptr<ml::Boost> boost = ml::Boost::create();
 boost->setBoostType(ml::Boost::REAL);
 boost->setWeakCount(10);
 boost->setMaxDepth(2);
 boost->setWeightTrimRate(0.95);

 cout<<"Training data: "<<endl
     <<"getSamples\n"<<dataset->getSamples()<<endl
     <<"getResponse\n"<<dataset->getResponses()<<endl
     <<endl;

 cout<<"Boostiing Model Trainning..."<<endl;
 boost = ml::Boost::train<ml::Boost>(dataset,0);
 cout<<"Finished Boosting Trainning!!!"<<endl;

 printf("Finished\n");

return 0;

}

And I got this

As you can see, the input data is pretty simple, just a four dimensions data. Alternatively, if I used:

Ptr<ml::TrainData> dataset = ml::TrainData::loadFromCSV("D:/testFile2/test.csv",0,0);

Then, everything is fine, no assertion failed occured.

Does anyone have idea about this? Thanks

Ok, thanks for your help. I will keep working on this, and post my answer here if I got one.

Hi Steven, thanks for your reply. I did build OpenCV3 with OpenCL enable. In the other way, I think OpenCL on Odroid is supported by OpenCL, because I can derived devices information by OpenCV API. Am I correct or not?

Hi all, I am trying to use OpenCV3.0 OpenCL API on my Odroid XU3. Unfortunately, I am encountering some real weird problems.

The following is my devices information which is derived by OpenCL 1.1:

PlatformCount: 1
1. Device: Mali-T628
 1.1 Hardware version: OpenCL 1.1
 1.2 Software version: 1.1
 1.3 OpenCL C version: OpenCL C 1.1
 1.4 Parallel compute units: 4
2. Device: Mali-T628
 2.1 Hardware version: OpenCL 1.1
 2.2 Software version: 1.1
 2.3 OpenCL C version: OpenCL C 1.1
 2.4 Parallel compute units: 2

I tried to using do some performance test on Odroid by OpenCV3.0 OpenCL API, so I wrote following code:

#include <iostream>
#include <string>
#include <iterator>
#include <opencv2/opencv.hpp>
#include <opencv2/core/ocl.hpp>

using namespace std;

int main(){
     if (!cv::ocl::haveOpenCL())
        {
            cout << "OpenCL is not avaiable..." << endl;
            return 0;
        }
        cv::ocl::Context context;
        if (!context.create(cv::ocl::Device::TYPE_GPU))
        {
            cout << "Failed creating the context..." << endl;
            return 0;
        }

    // In OpenCV 3.0.0 beta, only a single device is detected.
        cout << context.ndevices() << " GPU devices are detected." << endl;


    for (int i = 0; i < context.ndevices(); i++)   
        {
        cv::ocl::Device device = context.device(i);
        cout << "name                 : " << device.name() << endl;
        cout << "available            : " << device.available() << endl;
        cout << "imageSupport         : " << device.imageSupport() << endl;
        cout << "OpenCL_C_Version     : " << device.OpenCL_C_Version() << endl;
        cout << endl;
    break;
        }
    double tic, toc;

    cv::Mat mat_src = cv::imread("img_1.png", 0);
    cv::Mat mat_dst;

    cv::UMat umat_src = mat_src.getUMat(cv::ACCESS_READ, cv::USAGE_ALLOCATE_DEVICE_MEMORY);
    cv::UMat umat_dst;

    tic = (double)cv::getTickCount();
    for(int i = 0 ; i<1000; i++){
        cv::Canny(umat_src, umat_dst, 0, 50);
    }
    toc = (double)cv::getTickCount();
    cout<<"UMat X Canny Time: "<<(double)((toc-tic)/cv::getTickFrequency())<<endl;

    tic = (double)cv::getTickCount();
        for(int i = 0 ; i<1000; i++){
                cv::Canny(mat_src, mat_dst, 0, 50);
        }
        toc = (double)cv::getTickCount();
        cout<<"Mat X Canny Time: "<<(double)((toc-tic)/cv::getTickFrequency())<<endl;

    return 0;


}

and I got following results on my Odroid:

1 GPU devices are detected.
name                 : Mali-T628
available            : 1
imageSupport         : 1
OpenCL_C_Version     : OpenCL C 1.1
UMat X Canny Time: 44.6943
Mat X Canny Time: 44.0222

As you can see, the results are merely no differences whether use UMat or not. To ensure UMat really did some effect to performance, I try the exactly same code on my PC, and I got following results:

1 GPU devices are detected.
name                 : Intel(R) HD Graphics 4600
available            : 1
imageSupport         : 1
OpenCL_C_Version     : OpenCL C 1.2

UMat X Canny Time: 4.8207
Mat X Canny Time: 11.6929

The result showed that performance was dramatically improved by OpenCL.

So, why OpenCL API not work on Odroid? Is it related to the version of OpenCL?