Ask Your Question

advice for a hand tracking algorithm

asked 2015-05-06 12:40:20 -0500

Hello. I am currently working on a college project where I have to design a program for the AR.Drone 2.0 to track my hand with the front video camera and follow it. I am currently using OpenCV 2.4.10 and have managed detect the hand using a combination of skin and SURF detectors. The method is sufficiently accurate when the drone is hovering however it's too slow for tracking.

After my detection phase, I draw two bounding boxes around the hand: one that encloses the hand and another slightly larger one that becomes my region of interest. My thinking is that if the hand is fully enclosed by the bounding box in frame n, in frame n+1 it will have moved slightly beyond that box but will still be found within the second bounding box.

Currently I am tracking the hand inside the region of interest purely by applying skin detection and redrawing the bounding boxes based on the hand contours. It's very fast, however, when other objects with skin (such as my face, other hands etc) enter the region of interest it's impossible to distinguish between the two or more objects.

I am looking for some advice from your experience on a tracking algorithm for my hand that will work inside the bounding box and is not too computationally demanding (it has to work on live video) and somewhat easy for a beginner in computer vision to implement. From what I've read and seen on the internet I could try to use meanshift, camshift, FAST or something similar but I really don't know which one to try. I'm somewhat tight on time and I want to avoid experimenting with different algorithms only to see them fail so I am asking for your advice on this matter.

edit retag flag offensive close merge delete


As suggested below, you need to use a tracker in between series of detections and use the prediction function. That way if you loose a detection, your tracker will now where it went.

StevenPuttemans gravatar imageStevenPuttemans ( 2015-05-08 07:12:46 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2015-05-06 14:28:23 -0500

Amr gravatar image

not sure if I understood your problem as I am new to image processing. however the problem of tracking an object can be improved by having a state estimator like Kalman filter I searched the internet and found this may be it could help you

edit flag offensive delete link more


You said it right, "can be improved". In the sense the trajectory can be softened, but the main problem of tracking remains.

Gino Strato gravatar imageGino Strato ( 2015-05-09 02:21:51 -0500 )edit

That is right but with the kalman filter there is prediction phase you can rely on the previous motion (history) of hand and use it to predict model until the hand is found again but you need data association to reject wrong features. I tried on mobile robots localisation and when features are not present the kalman filter relies on the model for the robot motion until a feature is spotted again. I have not tracked by image I'm sorry but I suppose there should be something similar

Amr gravatar imageAmr ( 2015-05-10 12:24:41 -0500 )edit

I agree. In image processing there is no difference than in robot localisation. Anyway kalman filters relies on strong assumptions about the motion of the object (for example linear with constant velocity), techniques like particle filters are more general and fitted for tracking object inside images.

Gino Strato gravatar imageGino Strato ( 2015-05-10 15:13:21 -0500 )edit

Yes sure particle filter has advantages compared to kalman filter it can be used for multi hypothesis tracking and model non gaussian distribution but you need motion model still you can learn the motion of the object first using a classifier for example then pass that model to a filter for tracking with the assumption that the object motion is not varying rapidly for example human waving hand is fairly predictable motion also people walking around can be described by a learned motion model

Amr gravatar imageAmr ( 2015-05-20 06:05:47 -0500 )edit

Question Tools

1 follower


Asked: 2015-05-06 12:40:20 -0500

Seen: 1,172 times

Last updated: May 06 '15