If you look at the python2 lk_homography sample, you see that opencv can quite easily track the 2D perspecitve shift from one image to the next. If you open that sample and move your camera around, opencv can map exactly where the original image is in relation to the new one.
My question is really about whether it's possible to take this 2D perspective transform and turn it into a 3D one - e.g, if the points spread out in 2D, it's obviously coming closer to the camera in 3D - and if the points skew to the left or right, that's a 3D rotation.
Is there a function to do this? Something like cv2.getAffineTransform but in a 3rd dimension.
The end goal here is to use a camera to estimate change in position of a robot.