Evaluate the score of template matching when template's size equals image's size
Hello,
CONTEXT: I am doing a bit of image analysis for the project I am working on and I would like to identify some areas in my image. In those areas, are symbols I need later on so I want to find the black rectangles' positions for later use. Currently I have the black rectangle image and the mask to remove the interior parts of the pattern. I list all the contours of the image. For each contour I check if the size is approximately what I am looking for, if it matches I crop the original image using the contour rect, resize this crop to the template size and match the two. If the score from the match is suitable I accept it and add this contour to the list of black rectangles.
ISSUE: My problem is with the scores. I first struggle to determines what would be an appropriate threshold. When you have template size < image size, you get the best point, but in my case I get a score that is not directly apparently good/bad. Currently I am exploring SQDIFF, and so my idea was to make the worst (highest) score and us that as a scale (0 to worst) to judge my matches. For that worst score I did the following :
Mat imageWorstChannel = imageChannel.clone();
for (int i = 0; i < imageWorstChannel.rows; i++) {
Vec3b tempVec3b;
for (int j = 0; j < imageWorstChannel.cols; j++) {
tempVec3b = imageWorstChannel.at<Vec3b>(i, j);
if(tempVec3b.val[0] < 177) tempVec3b.val[0] = 255 - tempVec3b.val[0];
else tempVec3b.val[0] = tempVec3b.val[0];
if (tempVec3b.val[1] < 177) tempVec3b.val[1] = 255 - tempVec3b.val[1];
else tempVec3b.val[1] = tempVec3b.val[1];
if (tempVec3b.val[2] < 177) tempVec3b.val[2] = 255 - tempVec3b.val[2];
else tempVec3b.val[2] = tempVec3b.val[2];
imageWorstChannel.at<Vec3b>(i, j) = tempVec3b;
}
}
My problem is I gets some matche's scores above this supposedly "worst" score. Is there maybe something that I misunderstand on how OpenCV matches stuff? Is it the correct way to do what I want ?
ADDITIONNAL INFOS:
- I use imread (..., cv::IMREAD_COLOR) for my template and my image source and imread (..., cv::IMREAD_GRAYSCALE) for my mask
do you have an example image ?
what do you try to achieve with the code you show ? why is it manipulating the color image ? and which would that be ?
To my understanding when you use SQDIFF, you get a score that reflect the differences between the template and the source. Then 0, means it's a perfect match but you don't have a value for "bad" as it depends on the image size.
My idea was to find the worst/highest score possible so I could judge if one score is good (near 0) or bad (near the highest possible). What you see in the code is an attempt to create this highest score. the reasoning is, for each color (3 -> [0][1][2]) of a point, at worst the value will be either 255-value or simply value (depending if its below or above the middles values). Hence if the color is at (0,0,0) the worst possible difference is (255,255,255) or if the color is at (55,200,178), the worst is (200,200,178).
are you really trying to detect a black rectangle using template matching ? that seems a bad idea. (the whole problem looks artificial)sh
and the whole contour finding, is it only to match several rects ? (which probably should be done by nms suppression in the result image instead)
(BTW, thanks for taking the time to help me) Well, this is an example of a perfect image. In the really condition, I would like to use a video feed that might be less crisp and clear so I thought a more "fuzzy" capacity to detect pattern would be better. For the contour part, the idea was to avoid having to "scan" through the image and only test the relevant points of it. The contours seemed to be the easiest way to do that as they seems to list interesting area in the image.
My main issue here is with how the score is calculated, or at least to have a method to judge the "quality" of a score. I don't understand why my "find the worst possible" doesn't work but if there is a better way I'd be happy to use it.
and shoudn't you look at the result image from template matching instead, which already is a "xy map of probabilities"
What i would like to do is to compare two images T and I and get an appreciation of how much I is like T. With this appreciation, I can then use a threshold to say, ok this one is good enough, it must be what I am looking for and this one isn't so I don't add it to the list. Maybe template matching is not what should be used then ?
bad idea for template matching, which works on texture
i think, this is going nowhere. maybe you should focus on detecting your symbols, and , if still nessecary, use simple contours to find the rects (see samples/cpp/squares.cpp)
I found out about aruco while searching for a solution for this issue and what they do is : grey the image, threshold, then find contour with 4 points (among other things). I am currently thinking about using their lib/module and just modifying it to accept my type of pattern. For the template recognition, instead of using the template matching, what they do is, while knowing the number of squares in the pattern, they subdivided the pattern their looking up, then they just threshold to get the values (they(re using binary/black and white patterns). So I think, using the same methods (or even part of their codes), I should be able to do without pattern matching.