_Martin's profile - activity

Exactly :)

I guess you haven't read my second comment?

At the next step, the OpenCV cascade algorithm sets a STAGE threshold which guarantees the desired hit rate. So in case there is no single feature which already separates the samples in the way that already almost all positives are classified correctly (which should almost always be the case), you will always get a "stage HR" of 1 after learning the first weak classifier

In theory, the training does not always have to start with a HR of 1 (even though that might be the case in practice). And here is why: (instead of using the Gini index, I use the misclassification measure, because it's easier to understand) You start with training each weak classifier in the feature pool, so it yields a low misclassification rate (it does not try to yield a high hit rate). In case of decision stumps it searches for a threshold which separates the classes the best. After this step, the AdaBoost algorithm selects the "best" weak classifier (with the lowest misclassification). Usually (since those are "weak" classifiers) they might just be a bit better than 50% but most probably not in the high 90% of correct classification....

I'm not quite sure to have the answer, but I think I found something which might have something to do with it: in the above case, I used the Discrete Adaboost algorithm. And I do not get decision stump number 2 in case of Real Adaboost. So I'm guessing, since DAB does only classification {-1,1} and RAB yields to real numbers, only after reweighting the given samples, DAB yields to the weak classifier number 3.

Might that be it?

I dug a little deeper into the CV code of CvBoostTree and CvDTree and I got a slightly different explanation:

• A weak classifier is indeed trained by minimizing some misclassification function (in case of Real Adaboost, its the Gini index, see: CV doc). So the wanted Hit-Rate does not come into play in this part of the algorithm.

• It then takes the best feature (lowest misclassification) and uses this one as the next weak classifier in the current stage.

• As the last step in this iteration, the algorithm calculates the overall stage threshold. And now the defined Hit Rate comes into play: the stage threshold is calculated so that the desired Hit Rate is guaranteed.
• In the next iteration, the weights are modified according to their classification in the last iteration step, therefore yielding different weak classifiers (decision stumps).

If you take all of this in consideration, it is clear that a HR of 1 and FA of 1 are possible!

Ok, it seems like I used the wrong term "dummy", maybe "synthetic" would be the better choice. What I did was creating a pattern which I would like to detect. I overlayed that pattern on some background and therefore have a pretty good idea what the features could/should look like.

My main question is: how can it be that the both leaf values of a learned decision stump are equal? And how can it be that the same feature (with the same threshold) leads afterwards to different leaf values? Isn't the former weak classifier than redundant?

Hey,

I have a weird result using traincascade which I can't explained. I created a small set of dummy data just to get to understand what traincascade does.

I get the following results:

===== TRAINING 0-stage =====

<BEGIN
POS count : consumed   17 : 17
NEG count : acceptanceRatio    35 : 1
Precalculation time: 1
+----+---------+---------+
|  N |    HR   |    FA   |
+----+---------+---------+
|   1|        1| 0.428571|
+----+---------+---------+
|   2|        1| 0.428571|
+----+---------+---------+
|   3|        1| 0.142857|
+----+---------+---------+
|   4|        1|        0|
+----+---------+---------+
END>

and the created xml:

<stages>
    <!-- stage 0 -->
    <_>
      <maxWeakCount>4</maxWeakCount>
      <stageThreshold>2.4513483047485352e+00</stageThreshold>
      <weakClassifiers>
        <_>
          <internalNodes>
            0 -1 0 744.</internalNodes>
          <leafValues>
            9.0286773443222046e-01 -9.0286773443222046e-01</leafValues></_>
        <_>
          <internalNodes>
            0 -1 1 -1709.</internalNodes>
          <leafValues>
            -1.2098379135131836e+00 -1.2098379135131836e+00</leafValues></_>
        <_>
          <internalNodes>
            0 -1 1 -1709.</internalNodes>
          <leafValues>
            -1.4120784997940063e+00 1.4120784997940063e+00</leafValues></_>
        <_>
          <internalNodes>
            0 -1 2 3.5550000000000000e+02</internalNodes>
          <leafValues>
            -1.3462400436401367e+00 1.3462400436401367e+00</leafValues></_></weakClassifiers></_>
</stages>

From the first output, I'd say that the weak classifier #2 does not lead to better results. If you take a look at the learned decision stump in the xml output you see the following leafValues

-1.2098379135131836e+00 -1.2098379135131836e+00

which are exactly the same? So how does this weak classifier help in the classification task? I cannot explained what happens in learning here.

Testing the classifier by detecting in a random image leads to the exact same results, no matter if I use this weak classifier or not.

Can somebody explain this behavior?

I changed the grabbing of the negatives, by using another precropped vec-file as negative sample provider. This way I make sure, the same negatives are used.

I do not know yet where the randomness in training occures. I'll try to dig a little deeper into the code. But if somebody has an idea/explanation, I would be very thankful

I'm sorry to bother again. But when I take a look at the algorithm on wikipedia (http://en.wikipedia.org/wiki/AdaBoost...), the weak learner is selected by minimizing some error function (misclassification).

But (if I am right) this error should be (in general) higher for a weak learner with HR=1 and FA=1 than for one with HR=1/FA=0.98

What do I miss?

Tahnk you for your quick response, but in my case I made sure the same negative samples are used. So there is no randomness in choosing the negative samples.

There must be another reason! Does somebody know if there's some randomness in the boosting process?

Hey there,

I was trying to train a cascade classifier using traincascade. I used 2 different machines training on the same data with same parameters, but I seem to get different results. Since in my understanding there is no randomness to training/boosting the cascade, I wondered how i got these results?

Can anyone explain this? Or is there some kind of nondeterminism in the training process? Then I would be happy to get some deeper explanations.

Thanks for the explanation. I'm trying to dig deeper into the sourcecode and was looking into the boosting process, but somehow I got lost or at least I can't find where features are selected as weak classifiers. Any help?

One more question for my understanding: Wouldn't it be "better" to select a feature which already filters out some negatives? Or what is the gain in keeping those weak classifiers?

I am also wondering about this. Any ideas?