Dynamic gesture recognition

asked 2017-09-28 00:44:11 -0600

Mateusz
31 ●1 ●3

updated 2017-09-28 00:54:35 -0600

Hi there !

As my Master thesis project I have to design a dynamic recognition system using OpenCV. Could you please give me some piece of advice how to realise such a system in a quite robust way.

----This is done-----
- So first of all I want to detect the hand using haar caascades.
- Hand tracking region using Kalman Filter.
- Hand Segmentation.
----This is done-----
- (Dynamic) Hand gesture recognition - this is my problem. How to do it ?
Could I ask you to tell me how to realise this part. I've hard about HMM and neural networks. Are you able to give me some tutorials about this topic, maybe some pieces of code ?

Generally plase give me some suggestions what is you impression about such a system. Thanks in advance.

edit retag flag offensive close merge delete

Comments

it's somewhat unclear, what you have solved, so far

berak ( 2017-09-28 00:51:06 -0600 )edit

add a comment

2 answers

Sort by » oldest newest most voted

answered 2017-09-29 11:17:35 -0600

Pedro Batista

2213 ●5 ●18 ●42 https://lnkd.in/bw9XGH8

Generally plase give me some suggestions what is you impression about such a system. Depends on a lot of things.

How many different gestures do you want to detect?

How much time do you have?

How robust does the final application should be?

Can you gather gesture data?

Are there speed requisites?

If its a short amount of different gestures (2-5), there is no need to resort to neural networks or other rather complex machine learning algorithms. Given that you can already segment the hand, it should be easy to process it in such a way to differentiate between your small set of different available gestures. For example, if you need to differentiate between fist and a open hand, you can do that by counting amount of fingers, size of the hand, etc.

If your system needs to handle a lot of different gestures and you can gather gesture data, then it will be better to go for a machine learning approach, by recording examples of different available gestures and use it as training data for a classifier. You need a machine learning algorithm that can handle multiple responses. For a simple gesture problem I think a kNN or a Decision Tree will suffice, and should be a good way to start, since these are simple machine learning algorithms that can handle multiple responses. For more complex, faster and more accurate solutions, you can start looking at neural networks, eventualy deep learning, knowing that these solutions need more and more knowledge about machine learning and will also require more and more data to train.

edit flag offensive delete link

Comments

Ok I will give you more details. It will be done on Raspberry Pi 3, I should be ready with the project till the end of Janurary 2018, application should be quite robust, 8 on 10 gestures should be recognised properly. Yes I can can gather gesture data. It is forseen that I can use examples. It must be the system for simple gestures for light control in the room. For example when I move my index finger to the left then light turn on, when to the right then light turn off, when cross-wise then light brighter. It must be done using index finger. So you think that kNN is enough and machine learning is not necessary. But gestures must be dynamic, this is important.

Mateusz ( 2017-09-29 11:28:21 -0600 )edit

You can separate it to several pipelines. 1 : Segment out hands 2 : Use cnn to classify your gesture, even small cnn work very well on hand gesture classification(http://people.idsia.ch/~juergen/icsipa2011.pdf) 3 : Use tracker(KCF, mean shift etc) to track the hand, record their position/gestures 4 : Check the position of hand, trigger your action base on their direction/gestures

I have a pet project, pedestrian_flow_counter_by_opencv could serve as an example of step 3 and step 4