how to control the mouse using object tracking in ubuntu

edit

you probably want to read the faq

please come back, when you got any own effort to show.

Like @berak said and to add try these steps:

1. Create a color based object detector
2. Add the localisation retrieved from the detector to a tracker like a Kalman filter.
3. Use those positions to push the mouse coordinates. This will require you interaction with your operating system.