Matching HOG Images with OpenCV in C++

asked 2015-09-09 15:28:33 -0500

Leo_ gravatar image

I have currently implemented a way to calculate HOG descriptors using cv::HOGDescriptor() . I have done this for two images and calculated a similarity index between the two images using a basic distance measure.

A good match is something I define as a similarity value below 50. A perfect match has a similarity value of 0

However, when comparing these two almost identical images, I get confusing results (i.e. a high value). Below attached is my code and the 2 images.

Can anyone please tell me what is going wrong? i.e. Why is there a high similarity value when the images are very similar.

    #include <iostream>
    #include <opencv2/core/core.hpp>
    #include <opencv2/imgproc/imgproc.hpp>
    #include <math.h>
    #include "opencv2/ocl/ocl.hpp"
    #include "opencv2/highgui/highgui.hpp"

using namespace std;
using namespace cv;

int main()
   Mat image(imread("R2.jpg",1));
   resize(image, image, Size(64,128) );
   Mat img;
   cvtColor(image, img, CV_RGB2GRAY);

   Mat image2(imread("R2-roi.jpg",1));
   resize(image2, image2, Size(64,128) );
   Mat img2;
   cvtColor(image2, img2, CV_RGB2GRAY);

   vector<float> features;
   vector<Point> locations;
   vector<float> features2;
   vector<Point> locations2;

   HOGDescriptor *hog = new HOGDescriptor();
   HOGDescriptor *hog2 = new HOGDescriptor();

   hog->compute(img,features,Size(32,32), Size(0,0),locations);

   hog2->compute(img2,features2,Size(32,32), Size(0,0),locations2);

   Mat Hogfeat;
   for(int i=0; i<features.size(); i++)<float>(i,0);

   Mat Hogfeat2;
   for(int i=0; i<features2.size(); i++)<float>(i,0);

   double distance=0;
   for(int i=0; i<Hogfeat.rows; i++)
      distance+= abs(<float>(i,0) -<float>(i,0));
   cout << distance <<endl;

   return 0;

R2.jpg R2.jpg R2-roi.jpg R2-roi.jpg

edit retag flag offensive close merge delete



The problem is not with your code but with your theory. Your approach simply won't work, because images that are very similar to our human eyes can have very very different feature vectors. A simple shift like the one in your images will ruin a distance measure. Moreover, have in mind that you're working in a very high multidimensional space, and you're computing distance based on each of the components of the vectors in such space - if just one of your components differs a lot, your similarity value is useless. Same/similar objects can have vectors that are far away (in a distance sense), so that's the reason they're normally used together with Machine Learning algorithms that try to separate them from objects not belonging to the class.

LorenaGdL gravatar imageLorenaGdL ( 2015-09-10 04:41:16 -0500 )edit

HOG is not rotation invariant, that breaks your theory also :)

StevenPuttemans gravatar imageStevenPuttemans ( 2015-09-11 07:35:31 -0500 )edit

@LorenaGdL What would you suggest instead of HOG?

Leo_ gravatar imageLeo_ ( 2015-09-12 16:06:21 -0500 )edit

What are you trying to do, exactly?

LorenaGdL gravatar imageLorenaGdL ( 2015-09-14 02:48:40 -0500 )edit

@LorenaGdL basically I want to input an image with various different coins and I want it to identify each coin correctly and perhaps give a total value of the coins on screen. Basically I've segmented each coin using a hough transform, however how do I say that that coin (segmented) is actually a 10c or 50c etc?

Leo_ gravatar imageLeo_ ( 2015-09-20 04:01:14 -0500 )edit

@Leo_ in most countries you already have a difference in value based on material, so a basic color based segmentation and seperation between blobs from the Hough Transform should be able to solve it for you as a starter. A BoW implementation could be a follow up to differ in between same color coins.

StevenPuttemans gravatar imageStevenPuttemans ( 2015-09-21 02:09:41 -0500 )edit