Revision history [back]

You could try how the kalman tracker performs, based on your skin segmented hand regions.

http://docs.opencv.org/modules/video/doc/motion_analysis_and_object_tracking.html#kalmanfilter