Ask Your Question
4

traincascade detections with output score for precision recall curves

asked 2015-04-03 07:58:11 -0500

Hello everyone,

Like the title says I am performing cascade classifier classifications. On a given dataset I trained a set of models. My goal is to determine the influence of the amount of training data on the classifier output.

I have 5 models trained. All models were trained either until 35 stages were reached OR if the returned acceptanceRatio on the negative samples is lower than 10e-5 (because I want to avoid overfitting on the training data). So far so good!

  1. 50 positive and 140 negative samples --> 20 stages reached
  2. 100 positive and 280 negative samples --> 21 stages reached
  3. 250 positive and 700 negative samples --> 17 stages reached
  4. 500 positive and 1400 negative samples --> 35 stages reached
  5. 670 positive and 2000 negative samples ---> 35 stages reached

Then I use those trained models to perform detections on a validation/test set with the following command

vector<Rect> objects;
vector<int> rejectLevels;
vector<double> scores;
cascade.detectMultiScale( frame_gray, objects, rejectLevels, scores, 1.01, 1, 0, Size(96, 128), Size(118, 157), true);

Which actually also works since it starts detecting perfectly fine. Now in the past I have always used the returned double values to threshold my detection and to create different positions in the precision recall curves. This is a somewhat good measure to get the certainty of a detection as output score (which has been used by many OpenCV users). But here comes the problem now.

For model 1 and 2 the output values of the scores are ONLY 1 or -1, while the values of model 3 4 and 5 simply return a double value indicating the certainty.

Anyone has an idea what is going wrong here?

edit retag flag offensive close merge delete

Comments

Hmmm the downside to being a moderator is that people start to ignore our questions -_-

StevenPuttemans gravatar imageStevenPuttemans ( 2015-04-03 12:21:32 -0500 )edit
1

Maybe it is simply because if you are a moderator the level of your questions is out of the reach for the average user :)

Gino Strato gravatar imageGino Strato ( 2015-04-03 12:41:57 -0500 )edit

Possibly :)

StevenPuttemans gravatar imageStevenPuttemans ( 2015-04-03 15:30:51 -0500 )edit

1 answer

Sort by » oldest newest most voted
6

answered 2015-04-03 15:44:28 -0500

Gino Strato gravatar image

updated 2015-04-05 03:21:23 -0500

As you know, we can image this kind of classifier as a function which assigns a couple of values to every window it gets as inputs: a rejectLevels, that is the integer value representing the stage where it was eventually rejected, and levelWeights, the double value the boosting algorithm outputs (the one thresholded to pass the next level of the cascade).
The overloaded detectMultiScale(…)only considers and gathers the windows that reaches the last 4 stages (source code: if( classifier->data.stages.size() + result < 4 ).

What you experienced depends only on the little number of samples used to train the classifier.
In such a situation it could happen that just one weak classifier per stage can separate negatives from positives.
If so only 2 values are assigned by each stage: -1.0 and +1.0 and every threshold between them can separate perfectly the two groups.
Hence you get either a +1, when the sample is classified as positive (that is, it passes through all stages and the final one as well, keep in mind that there are many errors), or a -1 (stage last-1, last-2, last-3 and not passed).
This explains also the reason why model 3 needs less stages to train: some stage requires more than one weak classifier and so it does a better job compared to the ones of model 1 and 2.

edit flag offensive delete link more

Comments

O didnt look at it that way! Explanation seems reasonable! Thank you!

StevenPuttemans gravatar imageStevenPuttemans ( 2015-04-03 15:52:28 -0500 )edit

You can get a confirmation inserting the following lines:

   for(...; i++ )  
    cout << " l:" << reject_levels[i] << " w:" << scores[i];

Note than the last stage is always +1, while the others are -1.

Gino Strato gravatar imageGino Strato ( 2015-04-04 02:17:48 -0500 )edit

Question Tools

3 followers

Stats

Asked: 2015-04-03 07:58:11 -0500

Seen: 907 times

Last updated: Apr 05 '15