How can I get original coordinates after stereo rectification?

edit

I'm not sure what "stereo" part does, but for cvInitUndistortRectifyMap you can try following (Python code):

To remove distortion (transform points like cvInitUndistortRectifyMap transforms image)

pts_rectified = cv2.undistortPoints(src=pts_org, cameraMatrix=cameraMatrix, distCoeffs=distCoeffs, R=cameraMatrix)

It uses iterative algorithm with only 5 (hardcoded) iterations - error is subpixel, but it's not perfect.

To apply distortion

This direction is precise (no need for iterations). Maybe there is simpler way to call it.

Convert rectified 2D points to 3D points like:

(x-cx, y-cy, f)

In Python it could be:

pts_tmp = cv2.convertPointsToHomogeneous(pts_rectified)  # add column z=1
pts_tmp[:, :, 2] *= cameraMatrix[0, 0]  # z=1 -> z='f'
pts_tmp[:, :, 0] -= cameraMatrix[0, 2]  # substract cx and cy
pts_tmp[:, :, 1] -= cameraMatrix[1, 2]

then:

pts_org_back = cv2.projectPoints(pts_tmp, rvec=(0, 0, 0), tvec=(0, 0, 0), cameraMatrix=cameraMatrix, distCoeffs=distCoeffs)