3D reconstruction (especially SFM algorithms) are often related with pinhole camera models. The state-of-the-art of these SFM techniques is to look where the rays of 2D-3D correspondences in two different cameras intersect in object space. This enforces that the camera model is a pinhole model (where the 2D-3D ray is just a straight line).
But often in real world there are multiple lens system used, where you can't really figure out the ray of 2D-3D correspondence.
My question is: How does the SFM technique works with such multiple lens camera systems?