Crash when using image pyramid for matchTemplate
I want to improve the speed of cv::matchTemplate by following the image pyramid approach which was tried here and presented on the linked page which is currently down but can be seen via google-cache:
My problem is that the C++ implementation visible in google-cache crashes for specific image sizes. I'm on Ubuntu 14.04.3 LTS 64 bit and see the crash on both newest versions of OpenCV 2.4.11 and 3.1.0. I created a working (=crashing) implementation which is very close to the linked tutorial. These are the example images that cause the crash: reference.png, template.png.
reference.png is 145 x 128 and
template.png is 24 x 47
The line causing the problem is:
ref(r + (tpl.size() - cv::Size(1,1)))
It seems like the created ROI is 1 pixel too big but I don't know how to fix this correctly or if I'm doing something wrong.
Many thanks for any hint!
// main.cpp:
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
/**
* Function to perform fast template matching with image pyramid
*/
void fastMatchTemplate(cv::Mat& srca, // The reference image
cv::Mat& srcb, // The template image
cv::Mat& dst, // Template matching result
int maxlevel) // Number of levels
{
std::vector<cv::Mat> refs, tpls, results;
// Build Gaussian pyramid
cv::buildPyramid(srca, refs, maxlevel);
cv::buildPyramid(srcb, tpls, maxlevel);
cv::Mat ref, tpl, res;
// Process each level
for (int level = maxlevel; level >= 0; level--)
{
ref = refs[level];
tpl = tpls[level];
res = cv::Mat::zeros(ref.size() + cv::Size(1,1) - tpl.size(), CV_32FC1);
if (level == maxlevel)
{
// On the smallest level, just perform regular template matching
cv::matchTemplate(ref, tpl, res, CV_TM_CCORR_NORMED);
}
else
{
// On the next layers, template matching is performed on pre-defined
// ROI areas. We define the ROI using the template matching result
// from the previous layer.
cv::Mat mask;
cv::pyrUp(results.back(), mask);
cv::Mat mask8u;
mask.convertTo(mask8u, CV_8U);
// Find matches from previous layer
std::vector<std::vector<cv::Point> > contours;
cv::findContours(mask8u, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
// Use the contours to define region of interest and
// perform template matching on the areas
for (size_t i = 0; i < contours.size(); i++)
{
cv::Rect r = cv::boundingRect(contours[i]);
// This is the problem which leads to the crash if not catched here:
cv::Rect roi = r + (tpl.size() - cv::Size(1,1));
assert(roi.x + roi.width <= ref.cols);
try
{
cv::matchTemplate(
ref(r + (tpl.size() - cv::Size(1,1))),
tpl,
res(r),
CV_TM_CCORR_NORMED
);
}
catch (...)
{
std::cerr << "EXCEPTION!" << std::endl;
}
}
}
// Only keep good matches
cv::threshold(res, res, 0.94, 1., CV_THRESH_TOZERO);
results.push_back(res);
}
res.copyTo(dst);
}
int main()
{
cv::Mat ref = cv::imread("reference.png"); // 145 x 128
cv::Mat tpl = cv::imread("template.png"); // 24 x 47
if (ref.empty() || tpl.empty())
return -1;
cv::Mat ref_gray, tpl_gray;
cv::cvtColor(ref, ref_gray, CV_BGR2GRAY);
cv::cvtColor(tpl, tpl_gray, CV_BGR2GRAY);
cv::Mat dst;
fastMatchTemplate(ref_gray, tpl_gray, dst, 2);
cv::waitKey();
return 0;
}
Well, I notice that it's always starting from [1,1], rather than the 0,0 that would be the first pixel of the image.
I don't know if that's the fault of the boundingRect function not including the edge, or if the contours don't include the edge.