How to use the rotation and translation vector to translate undistorted image?

edit

Assume that I have

cv::Mat image;
std::vector<cv::Point> image_coordinates;
std::vector<cv::Point> corresponding_output_coordinates;

I want to generate an output image such that the coordinates matches.

Normally I think I can use

auto transform=cv::findHomography(image_coordinates,corresponding_output_coordinates);
cv::Mat output;
cv::warpPerspective(image,output,transform,{output_width,output_height});

but it seems that the images are distorted too. So I think I can use cv::undistortImage: (pseudocode)

Mat intrinsic;
Mat distCoeffs;
vector<Mat> rvecs;
vector<Mat> tvecs;
cv::calibrateCamera(
            <vector containing 1 element = corresponding_output_coordinates converted to 3d coordinate>,
            <vector containing 1 element = image_coordinates>, image.size(),
    intrinsic, distCoeffs, rvecs, tvecs);
Mat result;
cv::fisheye::undistortImage(image, result, intrinsic, distCoeffs);

Now what should I do with rvecs and tvecs so that the result is rotated and translated to the correct position?