Bus passengers detection and tracking using overhead camera
I need some advice on detection and tracking using OpenCv.
We have cameras inside a bus. Cameras locations could be changed. Main task is to know about each passenger, when he has entered the bus and when he's left. Here are some example frames from camera pointed to entrance: https://drive.google.com/open?id=0BwI...
I've tried to use background subtraction on entrance area to detect human blobs. But some people have, for example, gray-colored clothes that don't much differ from background. Also when passengers carry something like umbrellas or large bags - false detections appear. I thought about using something like haar or lbp cascades for detection, but passengers appearance differs very much (and should be kept in mind that they will look much more different in winter time).
For tracking I've tried using CamShift on HS frame. Sometimes people have low saturation - CamShift fails. Also it fails when HS - coloring of passengers has small difference to background.
Thought about using some classification/segmentation algorithms to differ areas containing different passengers from each other. Tried K-Means to make clusters containing passengers by X,Y - coordinates and Hue-Saturation - it does good clustering, but takes very long time (I need almost real-time performance). Tried watershed algorithm, but it's not very accurate.
Would be very appreciate if you will point me to some methods or researches or someting helpful.
There are no existing methods that may achieve the expected results of this project.
Even if you could create a method that would perfectly detect and segment all passengers despite environment conditions, you cannot guarantee that a passenger coming in will have the same appearance as when going out. What if during the bus trip he wears hat or jacket, changes hair style?
Kudos for trying a lot of different things, but I really think this is a dead end project.
One cannot change his appearance in one frame, so appearance changes could be tracked. If Camshift is used, histogram recalculation should help to achieve this. We don't need 100% accuracy, 80% would be ok)
If 80% accuracy is enough then maybe you can get somewhere.
This is no ordinary task, even more so if the resources are limited and you need it to work @ real-time. You'll need dozens of hours of research, development and testing.
You are planning the project as a whole, I'd split the project into simpler problems. First worry about detecting individual passengers. You should look into ML algorithms. Haar cascades won't give you the accuracy you look for in such complex environments. I'd start researching more powerful ML algorithms, such as Boosting and Neural-Networks. Deep Neural-Networks are the state-of-the-art of the technology, you should also look into that.
If you can complete that task with acceptable results in a reasonably big test dataset you can go into passenger tracking, where you'll face other problems, such as occlusion.
Hello! I have the same task: tracking passengers in a bus and I've tried to use background subtraction too but it's failed((Do You have somehow progressed in the solution of this task? Can you give me your video from camera in a bus?
hi @dimalert, please don't write answers, if you have none. thanks.