!! traincascade: So many people run in this issue: Train dataset for temp stage can not be filled

asked 2016-12-05 04:05:23 -0600

marcello gravatar image

I spent so many hours on this problem. Searching the openCV forum for solutions and also searching in stackoverflow and google. I tried all advices to fix this issue: "Train dataset for temp stage can not be filled.."

Some posts said that we have to use full paths in the negative.txt file rather than relative paths. Others said: That one has to take care that no \r or \n are supposed to be present in the negative.txt file (if you are working on windows). Or: Chaning parameters such as: -minHitRate and -maxFalseAlarmRate will help. Or: Increase amount of posiitives (or negatives)...I tried with 100, 200,...1000, 2000....... etc.....

I tried everything ! why does this program not provide a more informative error msg and why is there no general help for all many many users that run in this issue and wasting so many hours ?

edit retag flag offensive close merge delete

Comments

I may help you about this issue, but let me know what stage did you see that message? and what platform are you using? window or linux? what opencv version are you using?

Tim Pham gravatar imageTim Pham ( 2016-12-14 03:08:20 -0600 )edit

Hello Tim. I am working with windows OS and latest openCv version 2.4.13 (windows x64 ). Ususally training breaks on 2nd or 3rd stage. The object that I want to detect is very simple, it is basically only a shape. I assume that the features that are found are not related to the boundaries of this shape, it looks like the extracted features are related to regions inside the shape of concern. I want to detect objects of similar shapes.

This is the cmd:

opencv_traincascade -data classifier -vec "images\posVec.vec" -bg "imagesneg\neg.txt" -numStages 20 -minHitRate 0.99 -maxFalseAlarmRate 0.5 -numPos 2000 -numNeg 1400 -w 30 -h 90 -featureType LBP

Changing parameters such as numPos or numNeg or maxFalseAlarmRate may help to reach 1 stage more but it will never go further than stage 3

marcello gravatar imagemarcello ( 2016-12-19 01:36:29 -0600 )edit

As my experience, I would like you to change some points below

  • numNeg should be greater than numPos, for example, numPos = 600, numNeg=1500 => if you have 2000 pos images, you need 5000 neg images, if you have 1400 neg images, you need 560 pos images
  • the images path should be in full path, you should apply find command with full path, use full path for vector file as well. after creating negative and positive files, delete the last blank line in those files.
  • dont use LBP, as to my test, I have tested for 20 stages, but the 19th stage had been failed, version 2.4.x not use LBP, I recommend you not to use this option, otherwise your work will not be valuable

hope this help. Tim

Tim Pham gravatar imageTim Pham ( 2016-12-19 22:23:15 -0600 )edit

dont use LBP, as to my test, I have tested for 20 stages, but the 19th stage had been failed, version 2.4.x not use LBP, I recommend you not to use this option, otherwise your work will not be valuable

-_- seriously these kind of replies make no sense at all. The reason you did not get it to work is because you did something wrong. I am doing a PhD and each and every detector I make is with LBP features and works just fine ...

StevenPuttemans gravatar imageStevenPuttemans ( 2016-12-20 09:06:49 -0600 )edit

Hi Steven, are you sure LBP works fine on version 2.4.13? have you taken a look at the code of trancascade?

Tim Pham gravatar imageTim Pham ( 2016-12-21 22:09:34 -0600 )edit

Actually I am ... just confirmed...

StevenPuttemans gravatar imageStevenPuttemans ( 2016-12-22 03:12:56 -0600 )edit

Believe me ,LBP option will not work on version 2.4.13, I have debugged that option.

Tim Pham gravatar imageTim Pham ( 2016-12-22 03:28:12 -0600 )edit

@Tim Pham, seriously ... i have been using LBP features and traincascade since 2.1 and have not had a single issue with it. So believe me if I say you must be doing something wrong, just like all the other people in this forum who had issues with it.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-12-22 03:41:30 -0600 )edit

@Tim I don't think it's related with relative or full pathes since it the process is starting and working for some hours. It just stops at 3rd stage (Train dataset for temp stage can not be filled..). Same effect for HAAR (instead of LBP) @steven: Sure "we" are doing somewhere a mistake, that's why I am searching here for help. Probably the pos images are not "good" enough, I have a very simple shape in a 30x90 pixels image. Shape is white and background is black (Cannot improve this). Then I create samples according to:

opencv_createsamples -img MyShape.jpg -num 1500 -bg neg.txt -vec "./onesample/posVec.vec"   -maxxangle 0.15 -maxyangle 0.1 -maxzangle 0.1 -bgcolor 0 -bgthresh 51 -w 30 -h 90

I make 8 more vecs and merge them into 1 single vec. What can go wrong ?

marcello gravatar imagemarcello ( 2016-12-22 09:45:03 -0600 )edit

OK, Marcello, I have just pointed out some of my work experience that helped me get rid of the error, but I think my situation differs from you. it maybe your posNum and negNum are not good enough. yesterday, I have run full of 20 stages successfully. with 360 pos images and 900 neg images.

Tim Pham gravatar imageTim Pham ( 2016-12-22 18:42:15 -0600 )edit

Hi @steven, do you think the changing image width and height when create vector affect to detect result? as your mention earlier, size 80x80 is quite big. but my object window size is 80x80, does 20x20 size meet my requirement? Regarding to this LBP option, did you use -baseFormatSave option along with LBP?

and YES, @steven, you are right, LBP works just fine without -baseFormatSave.

@marcello, I appreciated about your "we", may I join to that ? lol

Tim Pham gravatar imageTim Pham ( 2016-12-22 19:05:14 -0600 )edit

What are the requirements of the neg images? Mine are 240x400, in color, I have created them by saving (each 10th) frame from a movie, thus they differ but some of them are very similar. Can this be a problem ? Should they be gray scaled ? what is recommended ratio compared to the pos images?

marcello gravatar imagemarcello ( 2016-12-23 01:41:32 -0600 )edit

I have read some post in this forum, the negative image sizes are free to collect. you can take them randomly.

Tim Pham gravatar imageTim Pham ( 2016-12-23 02:08:46 -0600 )edit

Let me wrap some things up, because we have 2 separate discussions here

  • If your object is 80x80 pixels, then you can still train a 25x25 classifier. The model size is just the reference of the smallest object you will detect. Also, in order to grab global features instead of specific local ones, one might argue a smaller model size is even better.
  • -baseFormatSave is a really bad choice, it is purely there for backwards compatibility reasons and transforms your model into some ancient OpenCV1.x version of the models ...
  • About the 3rd iteration stopping, can you supply us the full output of your training process? I got the idea some parts of your dataset are depleted.
StevenPuttemans gravatar imageStevenPuttemans ( 2016-12-23 04:14:33 -0600 )edit

(1 of 5)

opencv2.4.13\build\x64\vc12\bin>opencv_traincascade -data classifier -vec "./onesample/merge.vec" -bg neg.txt -numStages 15 -numPos 350 -numNeg 900 -w 30 -h 90 -minHitRate 0.995 -maxFalseAlarmRate 0.60 -featureType HAAR
PARAMETERS:
cascadeDirName: classifier
vecFileName: ./onesample/merge.vec
bgFileName: neg.txt
numPos: 350
numNeg: 900
numStages: 15
precalcValBufSize[Mb] : 1024
precalcIdxBufSize[Mb] : 1024
acceptanceRatioBreakValue : -1
stageType: BOOST
featureType: HAAR
sampleWidth: 30
sampleHeight: 90
boostType: GAB
minHitRate: 0.995
maxFalseAlarmRate: 0.6
weightTrimRate: 0.95
maxDepth: 1
maxWeakCount: 100
mode: BASIC
Number of unique features given windowSize [30,90] : 3533175
marcello gravatar imagemarcello ( 2016-12-23 10:05:19 -0600 )edit
===== TRAINING 4-stage =====
BEGIN
POS count : consumed   350 : 351
NEG count : acceptanceRatio    900 : 0.00205169
Precalculation time: 44.766
+----+---------+---------+
|  N |    HR   |    FA   |
+----+---------+---------+
|   1|        1|        1|
+----+---------+---------+
|   2|        1|        1|
+----+---------+---------+
|   3|        1|     0.35|
+----+---------+---------+
END
Training until now has taken 0 days 2 hours 25 minutes 8 seconds.

===== TRAINING 5-stage =====
BEGIN
POS count : consumed   350 : 351
NEG count : acceptanceRatio    900 : 0.000736583
Precalculation time: 46.154
+----+---------+---------+
|  N |    HR   |    FA   |
+----+---------+---------+
|   1|        1|        1|
+----+---------+---------+
|   2|        1|        1|
marcello gravatar imagemarcello ( 2016-12-23 10:12:39 -0600 )edit
===== TRAINING 6-stage =====
BEGIN
POS count : consumed   350 : 351
NEG count : acceptanceRatio    8 : 0.000417929
Required leaf false alarm rate achieved. Branch training terminated

Sorry for the ugly presentation.. This time i reached stage 5. however mostly it stops at stage 4-5 (with identical params) and usually it stops with "Train dataset for temp stage can not be filled"

marcello gravatar imagemarcello ( 2016-12-23 10:17:15 -0600 )edit

===== TRAINING 1-stage =====
BEGIN
POS count : consumed   350 : 350
NEG count : acceptanceRatio    900 : 0.0547778
Precalculation time: 43.747
+----+---------+---------+
|  N |    HR   |    FA   |
+----+---------+---------+
|   1|        1|        1|
+----+---------+---------+
|   2|        1|     0.08|
+----+---------+---------+
END
Training until now has taken 0 days 0 hours 45 minutes 40 seconds.

===== TRAINING 2-stage ===== BEGIN POS count : consumed 350 : 350 NEG count : acceptanceRatio 900 : 0.0277273 Precalculation time: 44.387 +----+---------+---------+ | N | HR | FA | +----+---------+---------+ | 1| 1| 1| +----+---------+---------+ | 2| 0.997143| 0.147778| +----+---------+---------+ END

marcello gravatar imagemarcello ( 2016-12-23 10:23:44 -0600 )edit

Thank @StevenPuttemans for your kind answer, indeed, you're really a senior .

I have another question. Do you think that 20 training stages are effective enough for an object detection? suppose, we have enough positive images and negative images (2000 and 5000), if we train for more than 20 stages, for example 36 stages , traincascade result will make more sense. Am I right?

Tim Pham gravatar imageTim Pham ( 2016-12-26 04:00:23 -0600 )edit

@marcello, why did you changed -maxFalseAlarmRate value?, it may be the cause of error, set the value to 0.5 and see how it goes..

Tim Pham gravatar imageTim Pham ( 2016-12-26 18:16:42 -0600 )edit

Hi Tim, in fact your advices (use HAAR instead LBP and changing maxFlalseAlarm..) helped me to reach stage 11. The classifier I get after stage 11 is quite poor in detecting my (very simple) object. Anyway thanks a lot for your help !

marcello gravatar imagemarcello ( 2016-12-29 07:32:34 -0600 )edit

congratulation ! finally, you got your thing, in spite of bad detecting result, you obtained the cascade file. for the detecting result, I guess you were wrong in creating positive images, I may help you for the next step, so, how did you create positive images?

Tim Pham gravatar imageTim Pham ( 2016-12-29 23:24:19 -0600 )edit