1 | initial version |
There is a fascinating paper by David Nister: An Efficient Solution to the Five Point Relative Pose problem
It is some time ago that I read it, but if I remember correctly, Nister could retreive 3d informations from a handheld camera in real time by comparing subsequent frames, finding matches between the frames and using them to compute the relative pose between the two frames. It's hard stuff to read, but it could be a start for you. More recent papers might have cited him, so maybe that's helping you, too.