3D object detection and tracking

edit

Hi,

Assuming that I have previously implemented a robust natural feature tracker for planar objects (i.e. pictures) using keypoint extraction, matching and pose estimation, how do I logically extend this to track 3D obects?

Assume that:

• Hardware is monocular camera (smartphone, but more likely desktop with attached camera during dev.)
• I control the lighting environment of the objects (so can limit specular, etc)
• The object is rigid
• The object has distinctive texture, and is against a distinctive background.
• I have digitized 3D models of the objects if required.

Both object detection and pose estimation is required.

There appear to be many tutorials on 2D NFT tracking on the internet, but none explains how to then extend this to matching keypoints against a 3D model.

To be clear, I'm not looking for a prebuilt solution (sure, Vuforia does this.) I'm looking for source-code or algorithmic-level information.