Is it possible to compare and match the keypoints and descriptors of an image (that we have already calculated) with a video that is running in the real time?

edit

Yes.

That's not too useful, so I'll ask what you have and haven't tried?

Have you tried finding the keypoints and descriptors from each frame, finding matches, and, if enough are found, drawing the rectangle? That would seem to be the obvious step.

Thanks a lot for replying! Actually, I'm just in the initial stages of my project. Whatever I said was just the approach which I planned to follow. I'm still going through the documentation of OpenCV. I have seen how SURF and SIFT work. But I am not sure if they would fit the output which I have in my mind. Could you please tell me if this is possible - I will just point my webcam towards an object, and the algorithm should identify which object it is and display the name of that object on one corner of the rectangle. Thanks again! :)

What you want is almost certainly in the features2d and xfeatures2d modules. Take a look at the descriptor matchers and the bag of words trainers.

Take a look and decide which meets your requirements. It's certainly possible, although you may need to take steps to make real-time. But first start with getting it to work on static pictures, then extend to video, then work to speed it up.

Hey Tetragramm! Thanks for replying and I'm sorry for this late reply. I started to work on my project. I am using OpenCV 3.0 configured to Visual Studio 2013. I am stuck in finding keypoints and descriptors of static images. Because SurfFeatureDetector, SurfDescriptorExtractor and BruteForceMatcher are unidentified and also the header file that used to include these identifiers, #include opencv2/nonfree/features2d.hpp is not there in version 3.0. Is there any other way of finding the keypoints and descriptors? If yes, please do share it. Thanks! :)

Well, BruteForceMatcher should be there, but it's been renamed to BFMatcher.

SURF is under copyright by the creator, so if you want to use that, you have to get it from them. You can use ORB instead, which is in some ways better than SURF. From THIS page, your choices are AKAZE, BRISK, KAZE, and ORB. There are other types in the xfeatures2d module, but ORB is your most likely choice to begin with.

Cool! I am thinking of going with ORB and BFMatcher. Thanks again for the info! :) Will get back if anything pops up! :)

Hi Tetragramm. Could you help me with this - I'm using C++ language for the project. I'm stuck in using ORB with BFMatcher. Detection and Extraction of descriptors of a single image is done. But when I am trying to detect and extract descriptors from two or more images and tried to use BFMatcher, I'm not getting the right idea on how to use ORB with BFMatcher. There are plenty of examples of SURF and SIFT with matching techniques but none with ORB.

Could you please provide with some sort of sample code implementing ORB with BFMatcher?

The only thing you should need to change is switching the Norm Type to NORM_HAMMING2.

Take a look HERE and you can see someone using ORB and the FlannMatcher. And HERE is one that actually does use the BFMatcher.

Thanks again! Will look into it! :)

Hey Tetragramm. So I'm in the part of finding matches between the frames in ongoing video and the images that I have in the background. Finding the keypoints and descriptors is done for the frames in video. Problem arises in how to store the keypoints and descriptors of images in the background. So my question is - 1. Does OpenCV provide support for databases (so that I can store the relevant info of the background images and perform matches)? If so, how can I link my program to the database to send and retrieve data? 2. Is there any alternative way to find the keypoints and descriptors from each frame and find matches with that of background images, instead of using a database?

Thank you! :)

I expect that you will want to store your background keypoints and descriptors on the disk somewhere and simply read them into memory when you start the program. This page shows how OpenCV can write and read data in a consistent format. http://docs.opencv.org/2.4/modules/co...

Thank You very much! :) Very helpful!! Now I can easily store and retrieve the keypoints and descriptors of my background images. :) I have this one more doubt. Say I have 5 images of 5 different objects A,B,C,D,E(.jpg's). I have found the keypoints and descriptors and have saved them in 5 different .txt files. Now lets say user gives an image of one of the objects as an input (might not be the same one as of the background images, say C_2.jpg). Now I find out the keypoints and descriptors of the image given by the user. Next when I compare the descriptors of .txt files and that of the input image (constructing a loop to first consider A.txt, later B.txt, .....) how can I say that the given input image is "this" object (here, object C).

To simplify, by considering what parameters can I come to a conclusion that "No this is not the right object, move on to the next one". In the example how can I code to tell that C_2.jpg is not object A, not object B, Yes! It is object C.

I'm afraid that's your job. I will help with specific questions like "What did I do wrong with this code?" or "Is there a built in way to do ____?" but I can't do everything for you or you won't learn.

Take a look at the documentation and samples and see if there's anything there. Look up research papers where someone else has solved the problem. Experiment.

Yeah you are right. I'm sorry. Actually the deadline to submit the project is nearing so was getting anxious and in bit of a hurry. It's true that I am an amateur in this field. But I'm learning something new everyday. Thanks to the internet and you :)

I have seen some methods like Homography, knnMatch and somethings related to training like Cascade Classifier Training. These are the options I've seen till now. Could you suggest which one to with. Or are there any alternative better solutions than the ones I specified? Thank you! :)