Ask Your Question

Track 3D Position of Blob/Fingertip

asked 2015-04-15 10:22:53 -0500

updated 2020-11-02 12:35:59 -0500

Hi there! I want to develop an app which will use an RGB-Depth sensor and track the the 3D Position of the fingertips connection(or generally a standard point around the connection) when a pinch gesture takes place.

image description

I have managed to get the contour,blob of the hand etc.However I want to get an accurate estimation of 3D position of the fingertips continuously when I move my hand. Any OpenCV tips?What to do/use?Any similar projects? Is it possible to get somehow the 3D Point on the fore-finger(index) somehow?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2015-04-15 13:22:36 -0500

Do you know this paper from Andy Wilson?

I implemented this pinch gesture with a kinct some years ago and (like wilson) used the center of the inner contour instead of the fingertips. If I remember correctly, I took the depth image, found the hole (your inner contour) and used a simple dilation (around 10px) of this contour as mask. By doing this, I got the 3d points on the border of the gap and used the mass of center as final Input Point. Andu Wilson proposes to do a pca on the pixels in the inner contour to get an orientation but this direction was rather instable in my experiments.

edit flag offensive delete link more


Hi Foobar.Yes I want to do something like that in the paper you mentioned. I am looking for a way to accurately move a virtual object in 3D space when I do a pinch. So you take the inner contour(hole) on the border of the gap-->which are hands pixels and then calculate mass center.I want to do something like that but using as a reference the area where the 2 fingers touch each other if possible.What would you suggest.Also can you tell me more specifically what you mean by dilation?Did you find any solution for orientation?

marios.b gravatar imagemarios.b ( 2015-04-15 15:10:09 -0500 )edit

I just drew the contour onto an image an applied cv::dilate to increase it. I found no stable solution for the orientation. A small error in the orientation will already lead to jumps in your input which is not very pleasant for the user. The same is valid for the fingertip. You need something very reliable and an error under maybe 3mm to have a good user experience.

FooBar gravatar imageFooBar ( 2015-04-15 16:06:24 -0500 )edit

Question Tools

1 follower


Asked: 2015-04-15 10:22:53 -0500

Seen: 775 times

Last updated: Apr 15 '15