Ask Your Question

OpenCL target for DNN slower than CPU

asked 2019-02-20 03:30:27 -0500

versusvoid gravatar image
  • OpenCV 4.0.1
  • Intel Core i5-7400, compute-runtime 19.05.12254

Using example on HAND dataset.

CPU target takes about 850ms per frame, OpenCL ~1.4s. Measuring with time module as net.getPerfProfile() lies about inference times. Enabling OpenCL SVM does not help.

Is it simply how it is, or there is some magic to speed computations up? How come broadly advertised (by Intel, no blame on opencv) GPGPU is actually slower than vectorized CPU version?

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2019-02-27 06:32:15 -0500

versusvoid gravatar image

updated 2019-02-27 06:48:31 -0500

Yes, it simply is how it is. OpenCV does pretty good job optimizing all DNN operations with SIMD instructions. These optimized operations turns out to run faster than Intel's OpenCL implementation. Turning SIMD off in OpenCV makes CPU DNN backend run 30x slower than OpenCL.

For me OpenCV's CPU implementation is only ~20% (200ms) slower than Intel's Caffe fork with MKL-DNN, and this fork even produce incorrect results on HAND model. So we can safely claim that OpenCV is the fastest FOSS CPU DNN engine for this specific model.

edit flag offensive delete link more

answered 2019-02-27 09:29:00 -0500

dkurt gravatar image

For Intel CPUs you may try to build OpenCV with Intel's Inference Engine backend (Intel OpenVINO): Depends on model it may boost your model in x2 or x3 times.

edit flag offensive delete link more


reproduce a reasonable number of copies of the Materials internally and use the Materials solely for Your Project

any distribution of the Redistributables must only be as part of Your Project which must add significantly more functionality than the Redistributables themselves

You may NOT: (i) use, copy, distribute, or publicly display the Materials; (ii) rent or lease the Materials to any third party; (iii) assign this Agreement or transfer the Materials; (iv) modify, adapt, or translate the Materials in whole or in part;

No thank you

versusvoid gravatar imageversusvoid ( 2019-02-27 23:19:34 -0500 )edit

@versusvoid, What is this?

dkurt gravatar imagedkurt ( 2019-02-28 03:15:15 -0500 )edit

Question Tools

1 follower


Asked: 2019-02-20 03:30:27 -0500

Seen: 1,026 times

Last updated: Feb 27 '19