Ask Your Question

Performance of OpenCL via cv::UMat (v3.0 [dev])

asked 2014-05-27 15:33:23 -0500

AdamRossi80 gravatar image

updated 2014-05-27 16:09:12 -0500

I am using v3.0 of OpenCV, which is currently still under development. I observe surprising performance behavior when executing arithmetic operations using the cv::UMat class. Just to preface, I understand cv::UMat invokes operations seamlessly on the GPU via OpenCL (note, I observed the load/memory on the GPU increased using GPU-Z, which is a GPU information utility that provides resource usage, memory usage, etc. My machine has the following specifications: Intel Xeon (X5672) @ 3.2GHz, 24GB memory, NVIDIA GeForce GTX580, Windows 8.1 (x64), Visual Studio 2012 Ultimate (x64).

The code below provides an illustration of the experiments performed, which is characteristic of other proprietary code that cannot be provided:

// allocate host buffer
cv::Mat host1 = cv::Mat(10000, 10000, cv::DataType<float>::type);

// populate host buffer with random data
cv::randu(host1, cv::Scalar::all(-1e20), cv::Scalar::all(1e20));

// copy host buffer to device buffer
cv::UMat device1;

// execute arithmetic operations
cv::UMat device2;
for (int i = 0; i < 1000;)
    cv::multiply(device1, device1, device2); i++;
    cv::add(device1, device2, device2);      i++;
    cv::divide(device1, device2, device2);   i++;
    cv::subtract(device1, device2, device2); i++;

// copy device buffer to host buffer
cv::Mat host2;

The code above is a simplified version taken from a comprehensive test environment that allows for inspection of execution time relationships with respect to experiment variables (e.g., buffer size, number of arithmetic operations, etc.). Note, the code above exhibits the problem that will be described. The code was instrumented with timing functions from the Windows API (i.e., QueryPerformanceFrequency, QueryPerformanceCounter) to extract execution timing. Each step in the procedure is analyzed:

  1. Allocate host buffer
  2. Copy host buffer to device
  3. Execute arithmetic operations
  4. Copy device buffer to host

Allocate host buffer

First, the host buffer is allocated, the plot below illustrates the amount of time it takes to allocate an integer and float32 buffer as a function of size. The result seems reasonable.

image description

Copy host buffer to device

Note: in the following steps, only the float32 result will be shown, as this is consistent with the integer result.

The host buffer is copied to the device using the cv::Mat::copyTo method, the result seems reasonable.

image description

Execute arithmetic operations

Next, the arithmetic operations are invoked. This result is a bit surprising, as the expectation is that the slope of the device (GPU) line should be less than the host (CPU). The GPU line might have a larger bias- i.e., the host might appear faster initially given less operations, but the GPU should ultimately be faster given more operations.

Note: the plot illustrates timing for a 500MB buffer (11448x11448).

image description

Copy device buffer to host

This leads to the motivation for posting this issue to the group. The buffer transfer time to the host appears to be dependent on the number of operations performed!

image description

If anyone can offer some insight into this issue, it would be greatly appreciated! Thanks for your time!

-Adam Rossi

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted

answered 2014-08-25 04:15:43 -0500

jakobwilm gravatar image

Hi Adam, in these measurements, how did you ensure synchronisation between GPU and CPU? The last plot would be consistent with the hypothesis that there is no sync, and hence part of the operations are measured in your copy-to-host timings. Regards, Jakob

edit flag offensive delete link more

answered 2014-08-27 08:05:17 -0500

AdamRossi80 gravatar image

Thanks for your reply, Jakob.

I do not know when GPU synchronization is happening while using OpenCV. However, the timing of each step indicates there is some dependency on the parameters. For example, on the arithmetic operation step, it takes longer to invoke 800 operations than it does 600 operations -- and the time is roughly proportional, which makes sense. However, like you pointed out, these might be asynchronous operations, i.e., some of the time might be showing up in the next step (copy device buffer to host). However, even if the operations are happening during the buffer copy to the host, the total time to execute the arithmetic operations is 2 orders of magnitude greater than the CPU implementation -- this has me puzzled.

I was hoping someone close to the design/development of the cv::UMat / OpenCL functionality in OpenCV would recognize this question and offer some insight. However, it seems like this will require digging into the source code to determine the behavior.

Thanks again for your reply.


edit flag offensive delete link more

answered 2017-05-18 13:16:52 -0500


I am running into a similar problem with copyTo using the UMat container. Did you find any resolution?

k = 0;
for (int i = 0; i < y5_u.rows; i++) {
    if (i % 2 == 0)


y4_u, y5_u, and out are UMats. The sizes are 640x480 and smaller. This snippet takes approximately 15 ms which is more than twice as slow then when on the CPU. Any insights on a path forward would be greatly appreciated!


edit flag offensive delete link more

Question Tools



Asked: 2014-05-27 15:33:23 -0500

Seen: 3,274 times

Last updated: Aug 27 '14