Ask Your Question
1

Opencv 3.0 SVM crashes for large number of iterations [closed]

asked 2015-02-19 02:46:27 -0500

BT gravatar image

updated 2015-02-19 05:27:43 -0500

SVM::Params params;
params.svmType    = SVM::C_SVC;
params.kernelType = SVM::LINEAR;
params.C=10;
params.termCrit.type =TermCriteria::EPS+TermCriteria::MAX_ITER;
params.termCrit.maxCount =3000;
params.termCrit.epsilon = 0.001;
Ptr<TrainData> svm1;
Ptr<SVM> svm;
svm1= TrainData::create(Tdata2,ROW_SAMPLE,Clbl);     
svm=ml::SVM::create(params);
            svm->train(svm1,0);

This is the simplest example that fails. Works like a charm for almost all of my train data (Tdata2), except some. The error is an "Access violation writing location" error, on line 173 of svm.cpp, which is this piece of code:

 void calc_non_rbf_base( int vcount, int var_count, const float* vecs,
                        const float* another, Qfloat* results,
                        double alpha, double beta )
{
    int j, k;
    for( j = 0; j < vcount; j++ )
    {
        const float* sample = &vecs[j*var_count];
        double s = 0;
        for( k = 0; k <= var_count - 4; k += 4 )
            s += sample[k]*another[k] + sample[k+1]*another[k+1] +
            sample[k+2]*another[k+2] + sample[k+3]*another[k+3];
        for( ; k < var_count; k++ )
            s += sample[k]*another[k];
        results[j] = (Qfloat)(s*alpha + beta); // this is line 173
    }
}

The error can be reproduced with any kind of data, if it is forced to execute for a large number of iterations in order to achieve desired accuracy. However, opencv states nothing anywhere about a "maximum number of iterations".

Note that LibSVM gives me correct results on the same data with same epsilon, but it requires ~8K iterations, which then executes without a problem.

Any ideas on how to solve the problem? It is a very weird problem, since it seems as a bad programming kind of error. As a side-note, i use visual studio 2010 on win7 x64 [32gb ram]. The labels file is Clbl.jpg and the Tdata2 file is Tdata2.jpg. Don't be alarmed by the .jpg extension, just Donwload, save as, and rename to .xml and they are in the standard format that opencv is accepting! This particular dataset crashes for 2709 iterations on my machine. For 2708 its ok.

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by berak
close date 2015-05-04 02:57:30.599587

Comments

can you give some numbers , e.g. for which iteration count it fails ?

berak gravatar imageberak ( 2015-02-19 02:50:54 -0500 )edit

Its kind of different for every problem, but the 3000 iterations in my kind of problems, will almost surely fail. Please remember that LibSVM can easily solve the same problems which seem to require ~6-8K iterations.

BT gravatar imageBT ( 2015-02-19 03:22:45 -0500 )edit

can't reproduce, unfortunately. not even with 100000.

damn thing is, if it's UB somewhere, the real fault will probably be somewhere else, not on line 173

berak gravatar imageberak ( 2015-02-19 03:38:38 -0500 )edit

Can i possibly link you to a file (binary or xml) of my train data, if you think reproducing it will help you solve it? It's not that big, about 24198x2 of floats.

BT gravatar imageBT ( 2015-02-19 04:08:20 -0500 )edit

good idea !

2709 iterations (2708 is ok here, too):

OpenCV Error: Assertion failed (y == 0 || (data && dims >= 1 && 
(unsigned)y < (unsigned)size.p[0])) in cv::Mat::ptr, 
file e:\code\opencv\modules\core\include\opencv2/core/mat.inl.hpp, line 752

btw, value of C ? (i used 1)

berak gravatar imageberak ( 2015-02-19 04:32:57 -0500 )edit

its constant at 10.

BT gravatar imageBT ( 2015-02-19 05:11:49 -0500 )edit

if you have any first thoughts or want me to help somehow, by all means ask :) At first i thought it was a problem related to the LibSVM "cache size" parameter, but at the default value ( 100M) i have no problem in LibSVM...

BT gravatar imageBT ( 2015-02-19 05:33:25 -0500 )edit

if i debug it, i get the above mentioned exception .

for row 17957, we end up in svm.cpp,L539, kr.idx gets assigned a value of -1, thus leading to the exception in debug mode, and to UB in release.

unfortunately, the whole lru_cache stuff is way over my head, can't go further here, i guess.

berak gravatar imageberak ( 2015-02-19 06:10:19 -0500 )edit

Thanks a lot eitherway! Shall i post it as a bug then?

BT gravatar imageBT ( 2015-02-19 06:22:12 -0500 )edit

yes, please. consider making an official issue here .

looks like a programming err to me, unrelated to os/compiler/bitness

berak gravatar imageberak ( 2015-02-19 06:22:30 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
2

answered 2015-05-04 02:37:05 -0500

BT gravatar image

As of the latest opencv version, this crash does no longer happen! (i can no longer reproduce it, though not tested exhaustively. Also, i didn't test if the accuracy of the resutls is the same, only that there is no crash issue.) Many thanks to the people who solved this problem!

edit flag offensive delete link more

Comments

berak gravatar imageberak ( 2015-05-04 02:56:25 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2015-02-19 02:46:27 -0500

Seen: 770 times

Last updated: May 04 '15