SVM training much slower in version 3.0.0

asked 2015-08-05 05:17:04 -0600

LorenaGdL
2026 ●2 ●13 ●33

Due to some problems with version 2.4.12, I've been testing SVM training with OpenCV 3.0.0, and I've found it is much slower now with the new module.

To share some results, these are the times I got using the same training data and paramters in both versions, and training over 100 iterations to average:

1000 samples, 5000 features/sample: 0.11/SVM (2.4.12) vs 0.19/SVM (3.0.0) -> 72% slower
5000 samples, 5000 features/sample: 0.50/SVM (2.4.12) vs 0.95/SVM (3.0.0) -> 90% slower
5000 samples, 50000 features/sample: 4.9/SVM (2.4.12) vs 9/SVM (3.0.0) -> 83% slower

Not only that, memory consumption is quite higher in new version too. As you can see in the below images, 2.4.12 (fist one) consumes less memory and in a much more stable way than 3.0.0 does (second one). image description

So the question is... was this expected when rewriting the ml module? Is there anything that can be done (by the users and/or the devs) to improve it?

Comments

Like said in the other topic, could you try adding cv::ocl::setUseOpenCL(false);. Even if you do not use UMat some functions do a huge amount of checks at first time run, which yields a giant overload of memory consumption.

StevenPuttemans ( 2015-08-05 05:40:32 -0600 )edit

I had already tried that (I saw the Sobel question too), but in fact disabling ocl leads to worse results: 0.21 vs 0.19 in the case 1 described above, 1.05 vs 0.95 in case 2 and 11.33 vs 9 in case 3

LorenaGdL ( 2015-08-05 06:03:16 -0600 )edit

Hmm so it is actually benefiting from the openCL interface ... what other multithread options do you have enabled?

StevenPuttemans ( 2015-08-05 06:28:12 -0600 )edit

You mean like TBB? I guess none, I'm using a fair simple build and code. Build options are the same in both versions btw.

LorenaGdL ( 2015-08-05 06:41:11 -0600 )edit

I am always surprised how powerfull the pthread backend is. On my system that actually works faster than OpenCL and CUDA.

StevenPuttemans ( 2015-08-05 07:31:59 -0600 )edit

I re-built with TBB support, just to check, and nothing changed. I may consider implementing my own parallel_for loop or even using pthreads, though I don't feel very confident about it (I think I'm not that skilled yet...). Anyway, any insights/comments/suggestions about the performance decrease in the new ml module are welcomed

LorenaGdL ( 2015-08-05 08:58:27 -0600 )edit

add a comment

answered 2015-10-13 20:05:40 -0600

tuannhtn

1500 ●10 ●19 http://4fire.wordpress...

This may be a late answer, but I hope this can help someone who encounters the same problem. The reason why SVM training part is slow: It needs to compute the latent feature matrix from two big matrices (in this question 1000x5000 matrix for 1st case, 5000x5000 for 2nd case and 5000x50000 for the 3rd case) while multiplication big matrices is not an ordinary routine of OpenCV, even when it invokes parallel techniques as a silver bullet. I have noticed that the OpenCV's SVM module has a pre-computed option for using with pre-computed kernel matrix, but it does not implemented currently. Hence, for solving your problem to gain high speed performance, you have to write the code for computing the kernel matrix (for both training and predicting parts) by using a dedicated library (such as OpenBLAS) or parallel techniques (TBB, C++ threads, etc ...) and the code for doing SVM tasks (train and predict) by using LibSVM directly. I have followed this way and gain good performance with 12000 samples, each one is a 25000-D feature vector, on a 64 bit Windows 7 machine with 4Gb RAM. Following are some resource links for reference:

edit flag offensive delete link

Comments

Though every insight is always welcomed, my main question was why it is so much slower in version 3.0 compared to 2.4.x (I guess same issues about the kernels apply to previous versions too). I am sure that there were big and right reasons to re-write the ml module (because if it ain't broke, don't fix it, huh?), but one would expect the performance of a newer and upgraded version to be at least the same of the previous one, and the mentioned results clearly show a huge step back. Moreover, once OpenCV integrates the option to train and predict with SVM classifiers, having to use another library is pretty cumbersome (and you would likely have to deal with incompatibilities and so). Let's just hope the new ml module will improve as time goes by. I appreciate your answer anyway

LorenaGdL ( 2015-10-14 04:04:34 -0600 )edit

You are right for the part about why OpenCV 3.0 SVM is slower than that of its 2.4.12. However, about using other tools, I have a different point of view: for each task, there are several best libraries we can use and we all hope there exists an all in one best library but in fact this situation rarely happens. I learn from Matlab, one of the most successful applications, that it exploits almost all the best libraries, boost, lapack, tbb, blas, and more, for specific tasks. The down side of combining a bunch of tools together is the error prone, but even OpenCV and others are still not perfect now. When I started with OpenCV, I thought it supported big matrices multiplication, linear algebra operations, etc. but it does not and I have to use other libraries with it and they coexist well.

tuannhtn ( 2015-10-14 10:16:23 -0600 )edit

add a comment

SVM training much slower in version 3.0.0

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

SVM training much slower in version 3.0.0 edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

SVM training much slower in version 3.0.0