opencv_traincascade (neg)image reader parallelization
Hello, using traincascade, one soon realizes that the most time consuming part is getting (later stages) negative samples. Though, the question is: can be getting negative samples parellelized? Maybe on some basic level. What issues can arise from simply dividing negative samples to N groups (N cores I wanna use) and use TBB? (I know this is not very nice solution, but it can help a lot) I think, that exact number of negative samples is not that important, if this is considered an issue (it really doesn't matter, if we have 9582 or 10239 samples instead of 10000).
Thanks, Igor
Actually, I do think it is possible to parallize this step, however, it will not create 'unique' negative samples, since the samples selected in the different groups will have no relation whatsover, since they have been selected without knowledge of eachother. I have no idea how much this would influence the algorithm.
However, feel free to try out and to add your parallelization to the openCV framework!
Steven, thanks for quick answer. If there will be 'unique' images in groups, we should have the same behavior as current implementation, I mean samples should be unique. Maybe more instances of negative image reader would do the job (each one with different images)... I posted this question, because I think this is the biggest bottleneck in training.
It is indeed the bottleneck of the training phase. Though I do not have the skills myself to optimize it further. However, if you go ahead with tackling the problem, be sure to inform with @Maria Dimashova, she is actually the person who implemented the algorithm, so she could provide the best guidance I think.
Thanks, for contact. I'll probably dig deeper into the problem.
Still no progress here? Btw one of methods to erase duplicates is make second pass on sampled negatives, it wont take long at all comparing to sampling time.