Hello,
During the past year, I worked on cleanup and speed optimization of a moderately large non-OpenCV codebase; I accomplished roughly two orders of magnitude performance improvement. Aside from optimizing compute-intensive code (using algorithm rewrites and AVX2 optimizations), the biggest speedups were obtained using 1) reference counting images and 2) using memory pools.
Memory pooling is essential to obtain performance; with 30+ threads being active, allocation of images using a memory pool is essential to obtain good performance.
As there is quite some overlap in functionality between our codebase and OpenCV, it might be viable to refactor our codebase to become a layer "around OpenCV" and rely on OpenCV to provide most of the functionality. That represent a major refactoring effort - but it is only a viable solution if OpenCV supports memory pooling (or custom allocators in the mat class that actually work correctly).
I tried to find answers to the question "does OpenCV support custom allocators/memory pools", but the few posts I encountered probably don't apply anymore as the API has changed.
Can somebody please pitch in and let me know whether mat custom allocators/memory pools are supported?
Secondly, memory bandwidth now limits the throughput of our implementation. I presume that an OpenCV pipeline would have exactly the same issues. There's a possible solution for this: implement algorithms using cache blocking, possibly using hundreds of co-routines (a C++20 feature). Any folks that would be interested in developing this approach within the OpenCV framework?
Thanks, Karel