I'm currently writing a piece of software that uses various different modules of OpenCV (some examples are edge detection via Canny, filtering operators, optical flow and I have some own algorithms that work on the opencv matrices).
My question is, with the introduction of UMat in OpenCV3, is there any reason to still use Mat? Currently I'm still using Mat everythere (having only recently moved to 3.0), but trying out Farnebacks optical flow method, I realized it's much faster using the GPU speedup of UMat. But I love uniformity so at best I'll want to use ONLY UMat or ONLY Mat in my entire software. So I'm now thinking about using UMat everywhere so that I won't have to convert between the two of them.
Is this a good idea? Are there drawbacks of using UMat everywhere? I read of some cases where using the GPU actually led to a loss of speed. Do these problems still exist in the gold release?