ROI with stereo calibration
My task is to calibrate stereo camera system in the full resolution and then use a ROI of the image sensor for the rectification and stereo matching. The stereo camera system is very slow in full resolution (~10 FPS) and by only using a ROI it can achieve very good performance (>60 FPS). Because the ROI can be arbitrarily moved it is impossible to calibrate every possible ROI with the checkerboard.
Now when I calibrate in the full resolution of the camera system then I can't use this directly to use it to rectify the ROI of the cameras unless the cameras are read out in full resolution which would neglect the performance benefit.
To my understanding I need to apply transformations to the camera matrices that I would supply to initUndistortRectifyMap. By doing this I get some artifacts on the edges and sometimes the stereo block matcher can't find a match probably because the epipolar lines are not aligned. My guess is that the transformation should also be applied to the camera matrices that are supplied to stereoRectify.
There is another question here that is similar but deals with scaling of the resolution.