Ask Your Question
0

Can template matching be used to identify the exact image on the screen if the image is present multiple times on the screen?{ Opencv Java}

asked 2017-06-30 06:05:59 -0600

jas gravatar image

updated 2017-06-30 06:14:34 -0600

berak gravatar image

I would like to know if this is possible. If yes, a sample code would be great!

edit retag flag offensive close merge delete

Comments

yes, it is possibe, assuming it works for a single image (no rotation/scaling present)

the only difference with multiple instances is, that you can't simply use minMaxLoc() on the result image, but have to use your own non-maximim/minimum suppression.

berak gravatar imageberak ( 2017-06-30 06:10:54 -0600 )edit

Could you give me a sample code for it in java?

jas gravatar imagejas ( 2017-06-30 06:41:05 -0600 )edit

2 answers

Sort by » oldest newest most voted
1

answered 2017-06-30 08:09:06 -0600

berak gravatar image

updated 2017-06-30 10:45:05 -0600

first, you should look at the c++ tutorial for this ..

i do not have a java sdk around, so you'll have to take a Kotlin answer instead here;)

starting like this: our template image description

>>> var tem = Imgcodecs.imread("logo_2.png", 0)
>>> var img = Imgcodecs.imread("multi.png", 0)
>>> var res = Mat()
>>> Imgproc.matchTemplate(img, tem, res, Imgproc.TM_SQDIFF)
>>> var mm = Core.minMaxLoc(res)

now, if we were looking for a single instance, we would be done here, mm.minLoc would be the found position. (or, maxLoc, if TM_ccoefs was used)

for multiple instances, we have to locate the "valleys" (or, "peaks" with ccoefs) in res.

naively, you would traverse the result image, do a 1st order differenciation, and keep the minima/maxima, but that's unpractical in java, so let's try to threshold the result image, until only single points are left, like this

(believe me, there are points in that image, squint real hard ;):

image description

>>> var bin = Mat()
>>> var thresh = mm.minVal*1.05
>>> Imgproc.threshold(res, bin, thresh, 255.0, -1) // invert, since we're looking for minima
>>> Imgcodecs.imwrite("bin.png", bin) // for visualization
true
>>> var idx = Mat()
>>> Core.findNonZero(bin, idx) // get the positions
>>> idx.dump()
[130, 60;
 471, 177;
 234, 253]
>>> // done xD. now where's my cookie ?
edit flag offensive delete link more

Comments

Thank you, for the answer. However, could you explain the solution a bit detailed? I'm quite new to opencv. I'm not sure if adding this here is correct, but my problem is explained here: https://stackoverflow.com/questions/4...

jas gravatar imagejas ( 2017-06-30 09:32:05 -0600 )edit

not unless you highlight exactly, where you're stuck. (and i won't read SO questions, to understand you.)

maybe take a look at the c++ tutorial first.

berak gravatar imageberak ( 2017-06-30 09:50:24 -0600 )edit

update: we DO HAVE java code for this NOW ! (at least for single templates)

go here, press java button ;)

berak gravatar imageberak ( 2017-06-30 09:54:37 -0600 )edit

but again, your main problem seems to be: you have to read up a bit about image correlation in general. code won't help, without knowing the background.

berak gravatar imageberak ( 2017-06-30 10:00:16 -0600 )edit

I'm able to identify the multiple instances of the image.However, I would like to know a bit more on how you achieved the "exact" image match. Say for example there could be minor changes in the image that we are going to match. "For multiple instances, we have to locate the "valleys" (or, "peaks" with ccoefs) in res." A bit more focus on this line please. My template matching method is TM_CCORR_NORMED.

jas gravatar imagejas ( 2017-06-30 10:21:32 -0600 )edit
0

answered 2017-07-04 02:47:00 -0600

jas gravatar image

updated 2017-07-04 04:00:54 -0600

C:\fakepath\pso.png

I have been able to identify the exact images if they differ based only on the y co-ordinates. I'm unable to identify the image on the left.(All 3 images are same). Could you let me know the reason for this?

    MinMaxLocResult mmr = Core.minMaxLoc(resultMatrix);
    Mat bin = new Mat(toFindIn.rows() - toFind.rows() + 1, toFindIn.cols() - toFind.cols() + 1, CvType.CV_8UC1);
    double thresh = mmr.minVal * 1.05;
    resultMatrix.convertTo(resultMatrix, CvType.CV_8UC1, 255.0);
    Imgproc.threshold(resultMatrix, bin, thresh, 255.0, Imgproc.THRESH_BINARY_INV);

    MatOfPoint mop = new MatOfPoint();
    Mat idx = Mat.zeros(bin.size(), bin.channels());
    Core.findNonZero(bin, idx);
    mop = new MatOfPoint(idx);
    System.out.println(mop.dump());

Template Matching method used is Imgproc.TM_SQDIFF.

P.S: Read about template matching from the c++ tutorial you were mentioning :) @berak

edit flag offensive delete link more

Comments

^ok, touché.

berak gravatar imageberak ( 2017-07-04 04:50:12 -0600 )edit

I would select by thresh=mmr.minVal find the position of the template than assign 255 in resultMatrix area related to the template. Iterate this until mmr.minVal is small enough (in case of TM_SQDIFF)

pklab gravatar imagepklab ( 2017-07-04 05:38:40 -0600 )edit

I'm getting the mmr.minVal as 0 when i use the TM_SQDIFF matching method. I'm able to get the co-ordinates of the template using the minLoc.

The issue i'm facing with the code i have mentioned above is that i'm able to identify the images which differ only on the y co-ordinates. I'm unable to identify the object on the left.

link text

I have tried this.

jas gravatar imagejas ( 2017-07-04 08:35:37 -0600 )edit

@berak is there some tweak i can do to identify the 3rd image also?

jas gravatar imagejas ( 2017-07-05 06:11:27 -0600 )edit

@jas I can't see any reason because you can't identify 3rd image. Please update your 1st question with your current code, including image construction. I'm investigating on similar issue here.

pklab gravatar imagepklab ( 2017-07-05 14:41:17 -0600 )edit

@berak. your code works fine! There was an issue with my image. The image on the left had very minor difference with the one i'm comparing to.(Had to zoom in to 800% !!). Sorry for the inconvenience caused.

jas gravatar imagejas ( 2017-07-05 16:38:38 -0600 )edit

@pklab, could you share your investigation details on the issue you have mentioned above?

jas gravatar imagejas ( 2017-07-11 04:43:11 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2017-06-30 06:05:59 -0600

Seen: 1,300 times

Last updated: Jul 04 '17