Extracting features with opencv for image classification

asked 2015-05-15 04:01:01 -0600

eruru gravatar image

Hi everybody, I started to study computer vision and machine learning. I have a dataset of images and my goal is to extract the features with opencv3 (using different feature detection algorithms: Fast, GoodFeaturesToTrack, Mser, Star, Sift, Surf) and using the extracted matrix of features with an algorithm of machine learning for the images classification.

After a lot of search I've started with SIFT and SURF and I used the tecnique of bag of word for obtaining my final matrix. Now the problem comes with the other algorithms, because I've noted that, for example FAST, provide only a keypoints extractor but not a descriptor, so seem that I've to use one of following descriptor:

  1. SIFT
  2. SURF
  3. BRIEF
  4. BRISK
  5. ORB
  6. FREAK

But I have a lot a doubts...I've read here that BRISK / FREAK / ORB are not for image classification problem and, since they are binary descriptors, I can't use it with the implementation of bow in opencv because the kmeans works only with non binary descriptor. If I use, for example, only the SIFT descriptor with keypoint extracted with other algorithm (Fast, Star etc) the program works but has sense?I've search and it not seems good strategy (I report belove some lines of my algorithrm)

bowTrain = cv2.BOWKMeansTrainer(10)
detector = cv2.MSER_create()
sift = cv2.xfeatures2d.SIFT_create()
kp = detector.detect(img,None)
kp,des = sift.compute(img,kp)
bowTrain.add(des)

Is correct or there are other tecniques that I can use? Thank you for help.

edit retag flag offensive close merge delete

Comments

let's put it this way: BOW is using kmeans, thus you need float descriptors (like sift or surf) to make it work, irrespective of the keypoints used.

berak gravatar imageberak ( 2015-05-15 04:13:20 -0600 )edit

or you can transform the descriptors Mat:

cv::Mat floatDescriptors;
des.convertTo(floatDescriptors, CV23F);

This is in C++, I do not know how to do it in python :( But I have found this

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-05-15 06:22:38 -0600 )edit

Hi berak, thank you for answer. So I can use only SIFT or SURF descriptors but the combination (for example) FAST keypoints + SIFT descriptor does have any sense or is completely wrong?

eruru gravatar imageeruru ( 2015-05-15 06:50:53 -0600 )edit
1

Hi thdrksdfthmn thank you for answer, I've hear about the conversion to CV32F but i found for example here that is not a good tecnique because maybe it could be happen that there is a bad clustering.

eruru gravatar imageeruru ( 2015-05-15 06:57:57 -0600 )edit

imho, you can use any keypointdetector you want. do your own experiments, and see, what works best.

(but , most probably, sift descriptors work best with sift keypoints)

berak gravatar imageberak ( 2015-05-15 07:07:38 -0600 )edit

Hi berak, thank you for reply. But can I use this approach to understand which algorithm gives me the best classification for images? Suppose that I run 2 test with following combination SIFT kp +SIFT descriptor and FAST kp + SIFT descriptor. If the latter test show really bad result, how can I be sure that the the problem is due to FAST itself, and not due to an incompatibility between them?

eruru gravatar imageeruru ( 2015-05-15 08:15:26 -0600 )edit

@eruru I do not see the problem in converting to CV_32F and then back to CV_8U. Or could flann compute a small distance that in Hamming way is very long? Can you give some examples? :)

thdrksdfthmn gravatar imagethdrksdfthmn ( 2015-05-15 09:43:16 -0600 )edit