Hello dear OpenCV gurus,
I am a beginner with computer vision and need your help.
I need to detect all different types of german traffic signs (~600) and want to use OpenCV to do so.
In an Android and iOS App the user either takes a picture of a sign post or a single traffic sign where most of the picture is filled by that single sign (size of the longer side is 1536px).
Picture from mobile where vz151 should be detected
I want to ease the process of selecting the type of the traffic sign by detecting which sign is on that picture.
I have the official SVG files of each traffic sign, so I created PNGs for each with a transparent background and from those samples using opencv_createsamples with several background images using -num and -max*angle parameters, but training using those did not work well - and apparently using real-world images is a better source.
SVGs of traffic signs
I also have a database (which is small yet, but will grow fast in the near future) where for each traffic sign I have a number of real-world pictures with the manually set annotations where exactly in the picture the sign is located (x y width height - like opencv_annotation).
I created samples (.vec) (via opencv_createsamples using the annotations from the database in sizes 102x90 and 102x102 according to their ratio) and trained using a list of negatives (.neg) for the five signs vz114, vz151, vz205, vz267, vz306 - the ones I currently have the most real-world pictures of.
Picture from mobile of a background for a sign
First I tried using only backgrounds in which the signs might occur as negatives (in their original size 2448x3264px). As the identification did not work well I tried with those backgrounds downscaled to 816x216px and prepending downscaled versions (to 125px for the longer side) of the all the other signs to the negatives of each sign (so vz114 has the positives of vz151, vz205, vz267, vz306 and downscaled backgrounds as negatives, vz151 has the positives of vz114, vz205, vz267, vz306 and downscaled backgrounds as negatives and so on).
downscaled version of an annotated vz114 used as a negative when training vz151
downscaled version of an annotated vz151 used as a negative when training vz114
Log of training for vz114 (109 samples):
PARAMETERS:
cascadeDirName: classifier/vz114/
vecFileName: vz114_102x90.vec
bgFileName: vz114.neg
numPos: 102
numNeg: 1000
numStages: 20
precalcValBufSize[Mb] : 1024
precalcIdxBufSize[Mb] : 1024
acceptanceRatioBreakValue : 0.0001
stageType: BOOST
featureType: LBP
sampleWidth: 102
sampleHeight: 90
boostType: GAB
minHitRate: 0.999
maxFalseAlarmRate: 0.5
weightTrimRate: 0.95
maxDepth: 1
maxWeakCount: 100
Number of unique features given windowSize [102,90] : 2292195
===== TRAINING 0-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 1
Precalculation time: 10
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.015|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 3 minutes 27 seconds.
===== TRAINING 1-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 0.262055
Precalculation time: 10
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.04|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 6 minutes 53 seconds.
===== TRAINING 2-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 0.0441287
Precalculation time: 12
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 1|
+----+---------+---------+
| 3| 1| 0.025|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 12 minutes 8 seconds.
===== TRAINING 3-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 0.00567959
Precalculation time: 11
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.051|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 15 minutes 51 seconds.
===== TRAINING 4-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 0.00135636
Precalculation time: 12
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.036|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 20 minutes 34 seconds.
===== TRAINING 5-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 0.000214335
Precalculation time: 13
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.046|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 30 minutes 18 seconds.
===== TRAINING 6-stage =====
<BEGIN
POS count : consumed 102 : 102
NEG count : acceptanceRatio 1000 : 5.3652e-05
The required acceptanceRatio for the model has been reached to avoid overfitting of trainingdata. Branch training terminated.
Log for vz151 (290 samples)
PARAMETERS:
cascadeDirName: classifier/vz151/
vecFileName: vz151_102x90.vec
bgFileName: vz151.neg
numPos: 274
numNeg: 1000
numStages: 20
precalcValBufSize[Mb] : 1024
precalcIdxBufSize[Mb] : 1024
acceptanceRatioBreakValue : 0.0001
stageType: BOOST
featureType: LBP
sampleWidth: 102
sampleHeight: 90
boostType: GAB
minHitRate: 0.999
maxFalseAlarmRate: 0.5
weightTrimRate: 0.95
maxDepth: 1
maxWeakCount: 100
Number of unique features given windowSize [102,90] : 2292195
===== TRAINING 0-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 1
Precalculation time: 14
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.037|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 4 minutes 22 seconds.
===== TRAINING 1-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.207943
Precalculation time: 15
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.053|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 8 minutes 42 seconds.
===== TRAINING 2-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.0417938
Precalculation time: 13
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 1|
+----+---------+---------+
| 3| 1| 0.108|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 15 minutes 13 seconds.
===== TRAINING 3-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.011292
Precalculation time: 13
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.071|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 19 minutes 40 seconds.
===== TRAINING 4-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.00267857
Precalculation time: 13
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 0.104|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 24 minutes 34 seconds.
===== TRAINING 5-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.000681222
Precalculation time: 15
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 1|
+----+---------+---------+
| 3| 1| 0.142|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 32 minutes 46 seconds.
===== TRAINING 6-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 0.00016981
Precalculation time: 14
+----+---------+---------+
| N | HR | FA |
+----+---------+---------+
| 1| 1| 1|
+----+---------+---------+
| 2| 1| 1|
+----+---------+---------+
| 3| 1| 0.204|
+----+---------+---------+
END>
Training until now has taken 0 days 0 hours 46 minutes 54 seconds.
===== TRAINING 7-stage =====
<BEGIN
POS count : consumed 274 : 274
NEG count : acceptanceRatio 1000 : 7.80834e-05
The required acceptanceRatio for the model has been reached to avoid overfitting of trainingdata. Branch training terminated.
Here you find the vector samples, list of negatives, created classifiers and the log from opencv_createsamples and opencv_traincascade
The code in Android to detect each classifier in the 1536px picture the user took is adapted from javacv-android-recognize
Mat greyMat = new Mat(photoMat.rows(), photoMat.cols());
cvtColor(photoMat, greyMat, CV_BGR2GRAY);
int signSize = (int) (photoMat.rows() * 0.32f);
cascadeClassifier.detectMultiScale(greyMat, detectedObjects, 1.25f, 3, 1,
new Size(signSize, signSize),
new Size(4 * signSize, 4 * signSize)
);
I followed the opencv docs and several tutorials on the web, but seem to be stuck here. I tried lowering the sample size to 51x51 and also to 26x26 but the detection just got worse.
Currently vz114 and vz151 are found on pictures of each other.
Can you help me improving my classifier training setup so I can do a better detection?
- Is there some problem with my setup in general? Do you have advice on how to improve it?
- Do I just need more positive samples?
- I have read on some websites/posts that latest during stage 2 or 3 I should have multiple rows in each stage. What can I change so I get that?
- Most detections I have seen are meant to identify objects as a small part in a big picture, is there a problem as I want to detect the signs in such a big size?
- Is the bad detection rate related to some size / scaling problem?
- The training finishes very fast - does that mean something is wrong with the setup?
- The classifier files are very small ~10KB compared to a face classifier (1.5MB) - does that mean something is wrong with the setup or is this just due to the small number of samples so far?
Thanks in advance, Stephan