Detecting multiple instances of an object
I'm aware that this question has been asked over and over again and I thoroughly searched before asking this, I've looked at every post regarding the subject on the OpenCV forum and quite a lot on stackoverflow, yet none proved decisively helpful therefore here I am. I want to find multiple instances of an object in a photo of a textile material, which means that the objects can be deformed and rotated. To be mentioned that the kind of object searched will be different every 10 minutes or so.
[Offtopic] Before going any further, I would like to say that I love the community that this library succeeded in gathering. I've had only great feedback and found lots of helpful posts all around. I'm a 2nd year CS student, part-time working on research for a company to get myself through university. Being the only guy there that works on computer vision and not having anyone in my circle of friends whom I could ask anything related to the subject proved to be quite difficult, and having finally a feedback regarding the subjects has been amazing. I'm truly grateful.
[Back to the topic] Given the fact that the objects can be deformed, a simple template matching is out of discussion. I've also tried taking advantage of the contours yet that was not enough as some objects are simply to each other like in this case where the kind of object I'm looking for is the flower with red contour:
I've also tried using the features and RANSAC to find a single object, then remove that object from the scene and repeat the process yet it proves to be rather time consuming. Given the fact that the objects that needs to be detected in the scene will be different quite often, SVM+HOG is out of discussion.
And here I am, left without any idea so far and I was wondering if you guys have any regarding the subject and how should I approach it, I'd be glad to hear about it. Nevertheless I thank you for your time reading about my problem.
What language are you using?
I am using C++.
Any chance that you can share your entire "features and RANSAC" code?
Try a look at https://docs.opencv.org/2.4/doc/tutor... It should solve the limits on template matching related to scale and rotation of the choosen template
Seems like a lot depends on whether you want a general solution or a solution for this pattern.
For this pattern, seems like you could strip out everything that is not red. Then run findCountors. Now, you are left with the centers of all the flowers and the leaves of the red flowers. Now, for each contour, the area or perimeter of the contour can be used to determine whether it is a center or not. If it is a center, then look at the points in a given diameter from the center of the center, if you hit another red point, you know that must be a leaf on a red flower and that you have the center of a red flower. Count those. This approach should be immune to deformation, rotation, and translation and somewhat immune to scaling (depends on how big a diameter you choose to look).
How about doing some texture analysis, like fourier transforms and looking for repeating patterns?
@sjhalayka Sure thing! I'll post the code later today, or at the most, tomorrow!.
@procton Yea, I've looked into that but again I'm getting to be dependent of extracting feature and RANSAC, thing which didn't work out for multiple objects. I thank you for your answer though. I looked in that artcile before and didn't prove to be that useful.
@jpsitorino You gave quite an amazing answer. Well thought! Unfortunately I'm looking for a general solution. Still, you had quite an interesting idea there. I appreciate your help! Do you have any idea on how should I approach it as a general solution?
@StevenPuttemans Didn't think about that before. I'm gonna look now into fourier transforms, it seems useful as I'm working with repeating patterns actually. I'll come back with a feedback soon.
Another variant would be to build a library of shapes you want to count. Each shape is characterized by the perimeter/area of the contours returned by findContours (on the assumption that different items in an image will have different perimeters/areas to at least some degree).. When a query image comes in, you run findContours on it and then compare each contour to the library shapes. If the perimeters/areas match to within some margin, then you count it as a matched shape. This approach would be rotation and translation invariant and, depending on what you margins are, scale and deformation tolerant. Other much more complex approaches might include k-means/agglomerative hierarchical clustering (where you could also add color as a discriminator).
@jpistorino Did exactly that a while ago but the findCountours often found small parts of the whole contour or took into consideration the neighbor contour as part of another one. I was thresholding the photo, applied a morphology close, canny it and the call the findCountours function. Was I doing something wrong here?
@StevenPuttemans I wrapped my head the past day figuring how to look for patters after applying the fourier transform. Could you give me some heads up on it, please? How can I identify the patterns?
@Antarus I only know it is being used for that, I would have to dig into literature myself ... so cannot help you out there.