Hi,
I've got a little problem with the results of stereoRectify() when using the cv2 python module. I calculated the matrix R and the vector T using stereoCalibrate() for 2 converging horizontally aligned cameras (rotated inwards by 15 degrees each). R is the rotation matrix that makes the right camera's principal ray parallel to the principal ray of the left camera (when applied in the right camera's coordinate system). It rotates the principal ray by 30 degrees (clockwise if you're looking on top of the two cameras).
What I'd expect the matrices R1 and R2 (calculated by stereoRectify()) to be, is a rotation by -15 degrees for R1 and a rotation 15 degrees for R2 to make the principal rays parallel. But it's the other way round. R1 is a rotation by 15 degrees and R2 is a rotation by -15 degrees around the coordinate system's y-axis.
Why is that? What am I missing? Are these matrices not meant to be applied to the cameras but the (virtual) image planes? Note that the rectification does indeed work as intended and I get correctly rectified images when I pass R1 or R2 to initUndistortRectifyMap() and use remap() after that. I'm just trying to figure out what R1 and R2 actually mean and the documentation of stereoRectify does not really help.
I appreciate any hints. Thanks a lot.