Ask Your Question

training an SVM with 68 Point2f data.

asked 2017-06-01 04:06:34 -0500

antithing gravatar image

I am looking into the opencv SVM implementation, and have a couple of questions.

My data is the landmark points from dlib, which I have as std::vector<cv::Point2f> resultPnts;

this vector contains 68 * Point2f.

Each of my training samples is one of these vectors. So for example, with the label '1', I might have 200 vectors of 68 points, and the same for label '2'.

In the SVM example, the Mat for training is set up as follows:

int labels[4] = { 1, -1, -1, -1 };
float trainingData[4][2] = { { 501, 10 },{ 255, 10 },{ 501, 255 },{ 10, 501 } };
Mat trainingDataMat(4, 2, CV_32FC1, trainingData);
Mat labelsMat(4, 1, CV_32SC1, labels);

in my example, should I:

int labels[27200] = {1,1,1,2,2,2... }; // 400 * 68 labels
float trainingData[27200][2] = { { 501, 10 },{ 255, 10 },......  }; // 400 * 68 points
Mat trainingDataMat(4, 2, CV_32FC1, trainingData);
Mat labelsMat(4, 1, CV_32SC1, labels);

or is there a cleaner way?

Additionally, Is it possible to return the 'percentage of a label' ? For example, if the result is half way between '1' and '2' labels, it would return 50% for each. Or is it just a 'on' or 'off' classifier?


edit retag flag offensive close merge delete


hmm, you want to classify 400 sets of landmarks, not single points, right ?

berak gravatar imageberak ( 2017-06-01 04:11:43 -0500 )edit

yup! probably more, as i add more emotions...

antithing gravatar imageantithing ( 2017-06-01 04:16:00 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2017-06-01 04:33:50 -0500

berak gravatar image

updated 2017-06-01 04:36:07 -0500

ok, assuming, you get a std::vector<cv::Point2f> resultPnts; from dlib, you'd make a single 1d row Mat of those for both training and testing, and add a single label per landmark set:

Mat trainData; // initially empty
Mat trainLabels;
for each set of landmarks:
      std::vector<cv::Point2f> resultPnts = ...;
      Mat row(resultPoints);
      trainData.push_back(row.reshape(1,1)); // one flat row per set
      trainLabel.push_back( theLabel );

so, for 200 sets, you get a [96*2 x 200] data Mat, and a [1 x 200] labels mat. prediction is similar:

      std::vector<cv::Point2f> resultPnts = ...;
      Mat row(resultPoints);

while you cannot get a percentage confidence instead of the labels here, you can get the distance to the margin:

   float dist = svm->predict(query, noArray(), ml::StatModel::RAW_OUTPUT)
edit flag offensive delete link more


Thank you! much appreciated. Is the distance to margin similar in result to a percentage? Could i use it for 'percentage of a smile', for example?

antithing gravatar imageantithing ( 2017-06-01 04:45:41 -0500 )edit

not really. but it's : large negative distance == good prediction, and large positive distance == bad prediction.

see 2nd image here

berak gravatar imageberak ( 2017-06-01 04:48:41 -0500 )edit

Ok, after getting this running, I think actually need a different approach. Sigh. What I need is to train a model with multiple labels, using 68 points. Then, pass the same 68 points and return a percentage of each label. Eg, 60% smile, 20% eyebrowRaise, etc. SVM seems too binary for this purpose. What would you recommend? KNN? ANN? Is there a similar sample that i could look at? Thank you again for your time!

antithing gravatar imageantithing ( 2017-06-01 07:55:53 -0500 )edit

KNearest, maybe ? you could check the distances to the 3 nearest neighbours.

data setup is the same as for SVM.

i still think, you need to classify a "set", not single points, but maybe you don't need all 68 of them ? most of them are on the "outer hull", and never contribute anything meaningful

btw, do you ever "normalize" the points ? (like : subtract boundingbox tl() and divide by width / height ?)

(i forgot, if the dlib landmarks are in image or in boundingbox coords)

berak gravatar imageberak ( 2017-06-01 08:07:57 -0500 )edit

I do, yup. I have them normalized in relation to the tip of the nose. I am looking into KNearest, as well as ANN. The latter seems to give what I want, with getWeights, is that correct?

antithing gravatar imageantithing ( 2017-06-01 08:21:05 -0500 )edit

no, findNearest() gives you the distances

(oh, and with the ANN, you get the raw logit values in the responses array from predict(). but again, those are neither percents, nor real probabilities)

berak gravatar imageberak ( 2017-06-01 08:31:26 -0500 )edit

hmmm. IS there any functionality to do what i want? Or will I need to write a custom algorithm, eg, search each point for its nearest neighbour and match to a template that way? Thanks again for your time.

antithing gravatar imageantithing ( 2017-06-01 08:42:17 -0500 )edit

stop thinking about single points, that'll get you nowhere.

berak gravatar imageberak ( 2017-06-01 08:51:21 -0500 )edit

ok, so I have 200 'sets' of points. What is the best approach to compare a sample set of the same points, to these 200 sets? Is there anything suitable in openCv?

antithing gravatar imageantithing ( 2017-06-01 09:04:13 -0500 )edit

imho, machine learning is already the best approach. there are also shape context distances (shape module), which try to fit one set of points against the other, and then calculate some distance, but hmm.

maybe you have to ditch the landmarks, completely, and go e.g. for cnn with whole images

berak gravatar imageberak ( 2017-06-01 09:12:24 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-06-01 04:06:34 -0500

Seen: 145 times

Last updated: Jun 01 '17