Extended 3D tracking with ORB

edit

I am developing a fairly simple mobile AR application, which uses a basic hamming code marker detection. Now I want to take that a step further, and create a way to continue 3D tracking of the scene based on the initial 3D position of the marker. I want to create a feature similar to the Extended tracking option in Vuforia SDK (but since I have to use open source options, Vuforia won't work here).

So what are the steps I have to take to achieve this? I understand that I need some kind of feature detection algorithm - for this I think ORB is a good option, right? After I've found the feature points in the scene, how can I detect if these points are on the same 3D planar surface as the marker?