Tracking without colour

edit

Hello all, here I have a problem statement. I have a solution for object tracking with colour using HSV filtering. I do have another solution of sequential image subtraction to track moving objects. But the problem is If I have a person who is moving the object the model fails often because it tracks the person also.

So my concern is how may I track the particular moving object using Colour image OR Depth image? The environment is noisy because of several elements such as walking people and object shadow, etc. etc...

I cannot use colour filter because the object colour can be changed or it may be of multi colour.

Hope I made my problem statement clear. Any kind of sample code or suggestion is highly appreciated.

I would like to use different shaped object and different colour or multi colour.

Thanks, Prasanna