Detect sub-image(Irrespective of scale/angle) and crop that part from main image

asked 2016-08-08 07:46:18 -0600

M Fahad Minhas gravatar image

updated 2016-08-08 07:53:03 -0600

Hi everyone,

This is the issue of matching two images, one is the smaller one and other (main) image is the larger one.

Here, i see it can be done by Template Matching, but template matching cannot be of use here because it doesn't work with scaled or rotated images.

What I want to do is to look for template/sub-image in the bigger image and if it is detected, extract/crop that part from the main (larger) image.

As I am getting main image from camera and i have to look for my template in it, due to camera frame, it can be of different scale and orientation so i need to implement maybe SURF/SIFT and i have no idea how to use it with OpenCV for Unity.

I am attaching sample images.

Kindly help if anyone knows the solution/algorithm for it.

Main Image(Camera Frame)

Template

Here it can be seen that, images can have variance in their size and color. Irrespective of these, how can we detect template and crop that colored part from main image?

Please help.

edit retag flag offensive close merge delete

Comments

there is no "Opencv for Unity". (what are you talking about?)

berak gravatar imageberak ( 2016-08-08 07:53:30 -0600 )edit

Are you sure? You can search it in Unity Asset Store, there is a package for around $100 there.

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 07:56:40 -0600 )edit

Either way, is there any C# solution? I would gladly convert it in UnityScript if someone has done this kind of thing.

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 07:58:15 -0600 )edit

sorry, let me be more clear: there is no support from opencv for anything c# at all.

any wrappers you might see out there, are independant, 3rd party, might have there own warts, and the opencv devs won't take any responsibility for problems with those.

then, note, that SIFT/SURF are patented, and you'll have to get a license, if you intend to sell your software.

berak gravatar imageberak ( 2016-08-08 08:04:42 -0600 )edit

That's fine.

But are you talking on behalf of OpenCV itself? If not then can u kindly give the directions for the solution? I'm myself not here for exact code for it, i just needed guide lines, and i guess that is why we can ask a question here. Can you tell if we can do this with OpenCV? regardless of any platform?

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 08:10:44 -0600 )edit

then, due to one image being "textured", and the other not, you won't find many "matching" features between both.

maybe read a bit here ?

berak gravatar imageberak ( 2016-08-08 08:11:29 -0600 )edit

Yes you're right, but i have just seen SURF working, it maybe able to detect it even if we put colors and leave the borders.

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 08:13:58 -0600 )edit

But i want to crop that detected colored part.

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 08:15:04 -0600 )edit

I have found the half solution to it.

I can run a SURF algorithm and draw the keypoint in a new image.

Anyone knows from here how can we crop the detected part from Main Image? Is there any API in OpenCV which can make image out of SURF's detected location in main image?

I can attach current code if required.

M Fahad Minhas gravatar imageM Fahad Minhas ( 2016-08-08 10:15:36 -0600 )edit
  • if you knew the original coords of your roi, you can use the homography, to find that in the other image.
  • if you didn't, you'll proobably have to do something like contours finding in the neighbourhood of your matched keypoints, and use some heuristics, to find your roi then.
berak gravatar imageberak ( 2016-08-08 10:37:48 -0600 )edit
see more comments