# How to plot the Precision-Recall Curve (PRC) for a Cascade classifier correctly?

I am following OpenCV 3 Blueprints' Chapter 5 Object Detection for Industrial Applications by Steven Puttenams

https://www.packtpub.com/application-...

The training was successfully completed and I computed the F1 score to be approximately 0.86 as validated on the training dataset of 523 images (just to test if the training was okay).

Ex.

[root@cobalt workspace]# opencv_traincascade -vec npimages.vec -data output_lbp2/ -numPos 444 -numNeg 888 -bg negatives.txt -numStages 20 -featureType LBP -precalcValBufSize 2048 -precalcIdxBufSize 2048 -w 24 -h 24


However, my main problem is plotting the Precision-Recall Curve (PRC) curve, as suggested in Chapter 5, we could use the ff. to output the score:

detector.detectMultiScale(equalize, objects, levels, scores, 1.05, 1, 0, Size(), Size(), true);


But the score I'm getting have very small variations:

FILENAME    NUM_RECT    X   Y   WIDTH   HEIGHT  SCORE
/home/cobalt/Data/IMG_20160610_170847.jpg   1   190 287 68  68  -1.08848
/home/cobalt/Data/IMG_20160610_170925.jpg   1   186 294 68  68  -1.06534
/home/cobalt/Data/IMG_20160610_170957.jpg   1   189 286 68  68  -1.06534
/home/cobalt/Data/IMG_20160610_171038.jpg   1   191 289 67  67  -0.998512
... ... ..  ..  ..  ...
/home/cobalt/Data/IMG_20160610_205103.jpg   1   190 291 68  68  1.82761
/home/cobalt/Data/IMG_20160610_205106.jpg   1   190 291 68  68  1.82761
/home/cobalt/Data/IMG_20160610_205122.jpg   1   194 297 68  68  1.82761


And this gives rather small values for precision-recall:

Ex. coordinates.txt

0.00761905  0.00761905  -1.09
0.00763359  0.00761905  -1.08
0.00763359  0.00761905  -1.07
0.00766284  0.00761905  -1.06
0.00766284  0.00761905  -1.05
0.00766284  0.00761905  -1.04
... ..  ...
0.00952381  0.00761905  1.77
0.00952381  0.00761905  1.78
0.00952381  0.00761905  1.79
0.00952381  0.00761905  1.8
0.00952381  0.00761905  1.81
0.00952381  0.00761905  1.82


With the values above, I cannot get a decent PRC curve. I'm not quite sure what's the problem here.... Though, I can compute the Precision-Recall values using the detect_simple.cpp as I modified it here:

https://gist.github.com/melvincabatuan/45a0de3624e99a5c34d308d4a0b99b45


Ex. Output:

The number of true positives (TP) are: TP = 628.
The number of false positives (FP) are: FP = 185.
The number of false negatives (FN) are: FN = 7.
Precision = 0.772448.
Recall = 0.988976.
F1 Score = 0.867403.

edit retag close merge delete

Did you solve this problem? I'm obtaining the same results but I don't know why. Thanks

( 2016-07-17 12:09:45 -0500 )edit

Sort by » oldest newest most voted

Hello guys, keep in mind that the forum sometimes gets more than 100 new posts each day, so keeping track of ones interested for me can only be achieved if people put a notice. This can be done by adding the @ sign followed with my username here without spaces in between! I will take a look at this, based on the info @Pessanha24 sent me through the official github repo for the book. Solutions will also be posted there for future reference!

As a reply to the actual post

• Could you somehow provide me with your model, test data en annotations file?
• Why would the smaller variations be bad? It depends on how similar the test images are I guess.
• Did you use the provided script of the book for calculating the precision-recall values? Or did you reprogram it yourself?
• Can you take a look at the retrieved number of TP FP FN for each image and see if that resembles the detection output?

Happy to hear from you soon!

more

Also I notice you changed the condition loops quite heavily. Not sure why you did that, but the original code worked fine for me. Care to comment on that?

( 2016-07-20 03:15:28 -0500 )edit

Official site

GitHub

Wiki

Documentation