Revision history [back]

Hi.

I'm using OpenCV4Android to try out template matching on an android phone. Essentially what I'm doing is taking a picture of some object that I wish to detect, cropping it and saving it as a template.

I then use my android camera and a surface view to constantly get images from the camera. I am then applying template matching on each image, converting the image to a Mat first. However, when applying template matching I only get around 3-4 fps.

What I am essentially doing is this:

** mCameraMat = inputFrame.rgba();

            int matchMethod = Imgproc.TM_CCOEFF_NORMED;

            // mTemplateMat resized in terms of video size in prepareMediaRecorder.
            // Very hacky solution so need to fix it!
            int result_cols = mCameraMat.cols() - mTemplateMat.cols() + 1;
            int result_rows = mCameraMat.rows() - mTemplateMat.rows() + 1;


            mResult = new Mat(result_rows, result_cols, CvType.CV_32F);

            // Move this to a new thread.
            Imgproc.matchTemplate(mCameraMat, mTemplateMat, mResult, matchMethod);
            Core.normalize(mResult, mResult, 0, 1, Core.NORM_MINMAX, -1, new Mat());

            // Localizing the best match with minMaxLoc
            MinMaxLocResult mmr = Core.minMaxLoc(mResult);

            Point matchLoc;
            if (matchMethod == Imgproc.TM_SQDIFF || matchMethod == Imgproc.TM_SQDIFF_NORMED) {
                matchLoc = mmr.minLoc;
            } else {
                matchLoc = mmr.maxLoc;
            }


            // Draw a boundary around the detected object.
            Imgproc.rectangle(mCameraMat, matchLoc, new Point(matchLoc.x + mTemplateMat.cols(),
                matchLoc.y + mTemplateMat.rows()), new Scalar(TrackingActivity.r, TrackingActivity.g,
                TrackingActivity.b, TrackingActivity.a), 2);

**

Where mTemplateMat is the template bitmap image converted into a Mat object.

The bottleneck is on the line

Imgproc.matchTemplate(mCameraMat, mTemplateMat, mResult, matchMethod);

If I remove that line, I get around 25 fps, which is much more acceptable. I'd be fine with anything above 13-14. I understand that template matching is a very expensive process and doing it every frame can be costly. I have tried to do it every 20 frames, but it still slows down the processing considerably, and the end video looks worse as there is a constant transition from a smooth fps display to a low fps display.

What are my options in optimising matchTemplate? Any tips are much appreciated.