Tracking orientation of table tennis paddle

asked 2014-10-07 08:03:13 -0600

Hello.

I've started working on a 3D game of table tennis. What I want is the user to play the game with an actual table tennis, and the game needs to track the orientation\position of the paddle.

I am new to OpenCV, can someone point me into the right direction?

Considering the simple shape of a paddle and the fact that the entire area has the same color, what should I be looking for regarding tracking where the paddle is? And regarding orientation of the paddle, what algorithm do I need to use?

Thank you in advance, Catalin.

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2014-10-07 09:56:22 -0600

FooBar
2938 ●9 ●37 http://magazino.eu/

I think you should start with a marker based approach. My favorite marker in OpenCV is the asymetric circle grid that can easiliy be detected using http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#findcirclesgrid

You need to attach the circle pattern (e.g. http://docs.opencv.org/trunk/_downloads/acircles_pattern.png) to your paddle and then uses the OpenCV calib3d lib to find it's pose relative to the camera. You then have to find the transformation between the paddle and the marker, but this should be rather easy.

With this approach, you get a very accurate and rather reliable positioning of your paddle and can concentrate on the game.

You could of course also try to find the paddle without marker (markers are always somehow ugly) but this will take much longer and won't probably never be as accurate as the marker-version.

What camera are you using? Make sure that you don't have a rolling shutter since this will introduce a larger error as soon as the paddle is moving.

edit flag offensive delete link

Comments

Thank you for the response,

I've looked at the links but unfortunately I can't make any assumptions regarding camera parameters and I can't add a marker (I want other people to play the game).

However, I have no problem with using assumptions related to the shape of the paddle (somewhat ellipse) and the color of the paddle (black\red).

So if I can positively recognize the paddle, finding in what direction it is moving should be simple (even if I use the size to determine if it is moving back\forth).

The big problem for me should be to determine the orientation of the paddle.

Can you point me into the right direction for recognizing the object? And later I can focus on the orientation part.

Thank you, Catalin.

Catalin Toma ( 2014-10-08 05:07:25 -0600 )edit

Come think about it, if it is reliable, why not implement it with a marker on the paddle and if I get more experience, I will ask more specific questions. Thank you!

Catalin Toma ( 2014-10-08 13:23:11 -0600 )edit

If the answer helped you, please accept it or upvote. Thanks!

FooBar ( 2014-10-10 06:43:39 -0600 )edit

add a comment

answered 2014-10-08 06:00:06 -0600

Pedro Batista

2213 ●5 ●18 ●42 https://lnkd.in/bw9XGH8

This sounds like a very complex and challenging problem, so we should divide the big problem into smaller simpler challenges.

Your final goal is to know the position and orientation of the paddle at all times. So, the first thing we need to think about is how to detect and track the paddle itself. This raises some very important questions. What kind of camera are you gonna use? What will be its position in regards to the player? Are the environment conditions stable? Or there may be different lighting conditions? Shape and colour approaches wont provide enough information to solve your problem due to your high demands (knowing its position at all times).

Have you thought about using a kinect for that? It would provide depth information that would be absolutely vital for you and would eliminate a great deal of complexity. With that, you could use a skeleton tracking like approach.. it tracks the position and orientation of your joints, and it could be easily adaptable to add an extra "joint" (the paddle). This kind of algorithm makes assumptions about your body to detect all your joints, and then uses a kalman filter to track their position and orientation. I think this would be a good starting point.

edit flag offensive delete link

Comments

I will try to simplify my requirements, since I cannot make assumptions about camera\light conditions.

Regarding position, I only need to know in which direction the paddle is moving, since I will map this to my world coordinates and also the player will look at the screen to see where the paddle is (with some hand-eye coordination I think he can play the game reasonably well).

I am more concerned with actually recognizing the paddle and finding it's orientation (the way I see it, the only assumptions I can make is the color and the fact that it has a simple shape).

Do you think I should approach this by searching the image by color? I've also heard of model based approaches, where I have a database of paddles, but would that mean I need a pattern for every orientation possible?

Catalin Toma ( 2014-10-08 06:39:15 -0600 )edit

I am not understanding how the game works. Is the player in front of the camera? Or you cannot even assume where the player will be? Colour based algorithms are used for simple tasks and in very controlled environments (such has part inspection). Imagine the paddle is red and the user is playing with a red shirt, or behind is red wall? The detection wouldn't work and your game would fail. If you are building something that is supposed to be robust, colour is simple not the way to go. If you this is some kind of fun project you could start trying some colour segmentation and see how it would work. I'm guessing it would work for small movements, but would fail as soon as the player moves the paddle a little faster.

Pedro Batista ( 2014-10-08 07:26:21 -0600 )edit

I imagined the game to be played on a laptop,with a standard camera. I really hopped that color combined with the shape of the paddle would help me recognize it. I already know how to work out the 3d and physics part of the game, that was what I wanted to add to make it special: to play it with a paddle instead of the mouse.

Catalin Toma ( 2014-10-08 08:30:14 -0600 )edit

If that was possible, Nintendo wouldn't have developed an advanced sensor based motion detection system for Wii, and Microsoft wouldn't have the need to develop the kinect 3D sensor. I cannot imagine how you can achieve that task, but I am not an expert on the matter. Please don't let me steal your motivation. Good luck :)

Pedro Batista ( 2014-10-08 10:45:33 -0600 )edit

I mean, it might be possible to recognize the paddle in some situations using only colour and shape analysis, but I can't imagine a robust algorithm using only that information. What you should do is gather some images of players with paddles in the position you imagine the game would be played, taken out of laptop cameras on a set of different environments, and then start experimenting some processing algorithms. You will draw some conclusions as to the feasibility of the project fast enough.

Pedro Batista ( 2014-10-08 10:52:26 -0600 )edit

Well as I said, I have no experience with openCV,can you name some algorithms\tutorials for:

finding an area filled with a color (maybe with a tolerance)
finding an area matching a certain shape (I could go with finding imprefect circles)
how to get an area of interest from the video, so I don't constantly need to search the entire image

Catalin Toma ( 2014-10-08 13:17:59 -0600 )edit

add a comment

Tracking orientation of table tennis paddle

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

Tracking orientation of table tennis paddle edit

2 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

Tracking orientation of table tennis paddle