Fisheye to pinhole projection

edit

Hi,

I am working with artificial data, simulating a stereo pair of fisheye cameras. I would like to feed those to ORB SLAM2, which only works with pinhole images (there is an extended version, but I would like to keep using the basic ORB SLAM2). For that, I tried the regular rectification pipeline: fisheye::rectifyStereo, then fisheye::initUndistortRectifyMap and finally remap.

Images look more or less fine (straight lines are straight). However, these images are still not useful for feeding them into ORB SLAM2, because the epipolar lines do not go through the same objects on the images.

__Questions__

1 - Am I missing anything here? Perhaps this is not the correct way of doing this?

2 - Assuming I get the correct images, which camera matrix should I use then for the pinhole projection?

Thank you!