While using warpPerspective with openCL I was having a memory leak. Disabling opencl by using Mat or calling cv::ocl::setUseOpenCL(false) solved the problem. I think I've tracked it down to a line in ocl_warpTransform that uses convertTo from Mat to UMat. Here is code to replicate this. Bug?
Mat matM(Size(5, 5), CV_32F);
UMat M0;
matM.convertTo(M0, CV_32F);