At https://docs.opencv.org/master/db/d58/group__calib3d__fisheye.html, it says
If the fisheye distortion is and only is changing theta to theta_d, and after the distortion, use pinhole projection,
then a more reasonable formula is
That is, tan(theta_d) should be used instead of theta_d to calculate x' and y'.
Is this a approximation error here?