Ask Your Question
1

OpenCV 4.4 and multiprocessing on Windows 10

asked 2020-09-05 20:44:53 -0600

Vijaykam gravatar image

updated 2020-10-26 04:21:48 -0600

Hello OpenCV community,

I downloaded and compiled OpenCV 4.4 on my Windows 10 machine. It has been working great, except ...

I can't seem to get multiprocessing support to work (in Python - version 3.8.5). So far I have tried: multiprocessing.Pool, multiprocessing.Process and concurrent.futures.ProcessPoolExecutor. I believe on windows, these modules spawn a command shell to execute the workload. What I see is that the workload fails with the dreaded message: "ImportError: DLL load failed while importing cv2: The specified module could not be found"

Interestingly, all OpenCV python samples use multiprocessing.pool.ThreadPool module to process workloads. I tried that as well, and it works, but from looking at TaskManager, it seems to spawn only one instance of workload. I believe multi-threading is useful only when you have mix of I/O bound and compute bound workloads. My code is purely compute bound, so this does not help.

I can get things to work by executing in multiple Windows 10 shells, so I believe my OpenCV install is fine. I have a 12 core desktop and I would like to use all of them!

Your help would be greatly appreciated - I am at a dead-end in terms of looking online for help.

Regards,

Vijay

edit retag flag offensive close merge delete

Comments

Hello all,

So wondering if I am doing something wrong - and multi-processing on Windows 10 is indeed working with OpenCV4.4 - or if I am better off downgrading to an earlier version of OpenCV? If earlier version, which one?

Thanks in advance for your guidance.

Vijay

Vijaykam gravatar imageVijaykam ( 2020-09-16 17:01:54 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2020-10-13 16:52:03 -0600

Vijaykam gravatar image

Hello,

After a fair amount of digging, I was able to find the answer and a workaround. Posting here, to help out others.

Basically, multiprocessing support for OpenCV, with Python 3.8 is broken on Windows 10 (not sure if it is working for Linux - likely is). It has to do with the way Python 3.8 revamped how module extension dlls are loaded. So if you want to use multiprocessing for OpenCV on Windows 10, use Python 3.7 or earlier.

There is a somewhat painful workaround to enable multiprocessing with Python 3.8. It is described in detail, in the bug report here: https://github.com/opencv/opencv/issu...

Using this workaround, I was able to run my heavy processing modules on multiple cores.

Hope this helps.

Cheers!

Vijay

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2020-09-05 20:44:53 -0600

Seen: 2,203 times

Last updated: Oct 13 '20