Confidence scores for cascade classifiers (revisited) [closed]
Hi!
I have trained a couple of highly successful local binary pattern (LBP) cascade classifiers for object detection using OpenCV 3.2. So far I have used the classifiers sequentially and used the combined output as a "semi" multi-class classifier. This approach is flawed however, as an area can obviously be detected as multiple objects, which should be impossible. Because of this I am now interested in acquiring a form of confidence score for each detection by the individual cascade classifiers. With such a confidence score I could compare the overlapping regions of interest as proposed by each classifier and remove the regions with the lowest confidence score. I realize that this method is somewhat crude, but it is the best I could think of for the cascade classifiers supplied by OpenCV.
So far I have browsed a bit trying to find something that provides such a score, and I stumbled upon this thread that discusses the overloaded version of the detectMultiScale function (as found here). This discussion combined with another source makes me think that a sort of confidence measure could be calculated for each of the output Rect() objects (after non-maximum suppression) of detectMultiScale. In the last linked source the author quotes a paper that mentions a possible way of doing this:
"In the OpenCV implementation, stage_sum is computed and compared against the i stage_threshold for each stage to accept/reject a candidate window. We define the detection score for a candidate window as K*stage_when_rejected + stage_sum_for_stage_when_rejected. If a window is accepted by the cascade, we just K*last_stage + stage_sum_for_last_stage. Choosing K as a large value e.g., 1000, we ensure that windows rejected at stage i have higher score than those rejected at stage i-1." from http://vis-www.cs.umass.edu/fddb/faq....
Does this seem like a possible solution to my problem? If so, how could I go about implementing it?
PS: I know a multi-class SVM is better for this sort of a problem, so I have implemented that too. The reason I want the confidence measure is so I can compare my two implementations.
from the fddb faq:
i'm a bit afraid here, that you can only compare scores whithin a cascade, but not between different ones.
Would that be true if the cascades are trained using the exact same parameters? I know that each cascade will find different features to focus on, but the amount of stages that the sample passes through (if it does) should be equal for n m-stage classifiers?
cc @StevenPuttemans ;)
@berak@ThorbjornSomod, I have been breaking my head about this for ages. Someone should crack open the sourcecode and ouput the scores correctly for each window, even if it is not passed through the whole cascade (as it is now). Thats the only way you will ever find a decent evaluation of classifiers using precision and recall. However, it is not in, and the fddb guys never replyed to any of my queries of how they did it ... The book OpenCV3Blueprints suggests in chapter 5 how I have been doing it up till now, but it is still flawed ... Since I have been using other techniques lately, I never found the time to do it. Maybe after PhD is finished :D