OpenCL target for DNN slower than CPU

opencl
dnn

asked 2019-02-20 03:30:27 -0600

OpenCV 4.0.1
Intel Core i5-7400, compute-runtime 19.05.12254

Using openpose.py example on HAND dataset.

CPU target takes about 850ms per frame, OpenCL ~1.4s. Measuring with time module as net.getPerfProfile() lies about inference times. Enabling OpenCL SVM does not help.

Is it simply how it is, or there is some magic to speed computations up? How come broadly advertised (by Intel, no blame on opencv) GPGPU is actually slower than vectorized CPU version?

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2019-02-27 06:32:15 -0600

versusvoid
31 ●4

updated 2019-02-27 06:48:31 -0600

Yes, it simply is how it is. OpenCV does pretty good job optimizing all DNN operations with SIMD instructions. These optimized operations turns out to run faster than Intel's OpenCL implementation. Turning SIMD off in OpenCV makes CPU DNN backend run 30x slower than OpenCL.

For me OpenCV's CPU implementation is only ~20% (200ms) slower than Intel's Caffe fork with MKL-DNN, and this fork even produce incorrect results on HAND model. So we can safely claim that OpenCV is the fastest FOSS CPU DNN engine for this specific model.

edit flag offensive delete link

add a comment

answered 2019-02-27 09:29:00 -0600

dkurt

1424 ●7 ●17

For Intel CPUs you may try to build OpenCV with Intel's Inference Engine backend (Intel OpenVINO): https://github.com/opencv/opencv/wiki.... Depends on model it may boost your model in x2 or x3 times.

edit flag offensive delete link

Comments

reproduce a reasonable number of copies of the Materials internally and use the Materials solely for Your Project

any distribution of the Redistributables must only be as part of Your Project which must add significantly more functionality than the Redistributables themselves

You may NOT: (i) use, copy, distribute, or publicly display the Materials; (ii) rent or lease the Materials to any third party; (iii) assign this Agreement or transfer the Materials; (iv) modify, adapt, or translate the Materials in whole or in part;

No thank you

versusvoid ( 2019-02-27 23:19:34 -0600 )edit

@versusvoid, What is this?

dkurt ( 2019-02-28 03:15:15 -0600 )edit

add a comment

OpenCL target for DNN slower than CPU

2 answers

Comments

Links

Question Tools

Stats

Related questions

OpenCL target for DNN slower than CPU edit

2 answers

Comments

Links

Question Tools

Stats

Related questions

OpenCL target for DNN slower than CPU