Ask Your Question

finding icon on taskbar

asked 2014-11-12 15:25:42 -0500

hello I'm trying to find the position of my icon on the taskbar. I am able to grab the taskbar image and I have my icon image but can't seem to get a match. This seems like it would be easy but does have to be scale tolerable since the taskbar could be a different size.

I've tried every feature matching to no avail (seems icons are too small). I think the way would be to do a template match in a loop while increasing the size of the template?

not sure but has anyone tried this? seems opencv should have a way. (I've print screened the taskbar and created a template from the windows taskbar icon and saved them to disk for testing but can't get a match).


edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2014-11-13 08:51:45 -0500

R.Saracchini gravatar image

updated 2014-11-19 13:33:38 -0500

To find an icon with feature matching, it should have enough and distinctive features to be able to be detected. I must say that there are icons that wont be easy to detect with this approach.

Since your icon is not rotating or being distorted by perspective/affine projection, template matching seems me the best option for you. OpenCV has the matchTemplate function, which computes a similarity score for every pixel of the image, given a template. There is a good tutorial about this in the Tutorials section in this link.

If the icon size varies, you should try a multi-scale approach, that is, try to small templates, and increase its size until find a good detection score.


Well, sadly I cannot provide you working code for this, but I'll try to explain you the ideia. In fact if you intend to do anything serious in OpenCV, you should familiarize yourself with its basic structures (Point, Mat, Vec...).

Indeed, the example provided always detect something, even if it is not the most adequate match. A more robust solution, would be instead get the most likely region, get a set of most likely matches. That is, for the returned matrix "result" in the code of the tutorial, instead get the maximum value of the matrix (CV_TM_CCORR, CV_TM_CCORR_NORMED, CV_TM_CCOEFF, CV_TM_CCOEFF_NORMED) or the minimum value of the matrix (CV_TM_SQDIFF, CV_TM_SQDIFF_NORMED), you should scan the "result" for the N highest/lowest values on the result matrix. You can cut some off of such candidates by using some threshold.

This will give you a set of candidates. There is a reasonable way to identify if it is a good match or not. What you can do next is try to compute the photometric parameters for each candidate. Lets, say that this template has its pixel intensities multiplied by constant factor A, and a constant "background" brightness added B in the scene. That is, every pixel of the patch/template T on the scene S will have this form:

A(T[x,y]) + B = S[u,v]

Where (x,y) is the point of template T mapped to the point (u,v) in S. At first, a good candidate would be the one where A is closest to 1.0 and B closest to 0. Positive values of A and B means a shaded template (A < 1.0 -> darker, A > 1.0 -> brighter), where A and B < 0 are meaningless, thus a false match. Take into consideration that there can be some numerical errors due floating point representation in PCs, so A = 0.9998 and B=-0.01 still means a good match.

So, you must find the best value of A and B for each location candidate assembling a linear system using the template and the scene data. Note that, with the data from the images, this is a overdetermined system. Use Linear Least Squares) in order to solve it. OpenCV provides a linear system solver with cv::solve, so you can use ... (more)

edit flag offensive delete link more


i can't get it to find it. it actually matches the services icon. how would I know when to accept the match? the icon would not be rotated but could be a different size. anyone have some code I could try? sorry newbie questions with opencv.

jsojourn gravatar imagejsojourn ( 2014-11-19 09:41:55 -0500 )edit
Login/Signup to Answer

Question Tools


Asked: 2014-11-12 15:25:42 -0500

Seen: 287 times

Last updated: Nov 20 '14