Possible solutoin for marker-less wrist augmented reality

edit

The final goal is create a marker-less augmented reality app similar to this one

The solutions I come up are

1. Detect the location of wrist
2. Detect the wrist pose(similar to head pose of this one)
3. Obtain yaw, pitch, roll of wrist

I plan to do everything by cnn, but data collection steps could be a huge problem, do we have a better solution?