pose estimation from 2d-to-2d correspondences

edit

For explanation, assume that we have two image (image 1 and image 2) and camera intrinsic matrix K. We can get the correspondence between two image by local feature matching. What I want to get is 6DOF pose from image 1 to image 2.

To get pose, I got fundamental matrix F from 2D-to-2D correspondence. Then, I calculated essential matrix E from F and K. After decomposing E, finally I made projection Matrix P. I would like to transform Rectangle ROI in image 1 to ROI in image 2. I set depth of ROI in image 1 to zero. (0, 0, 0), (w, 0, 0), (w, h, 0), (0, h, 0) Then, to get ROI in image2, I used project matrix P. But ROI is strange. Is there anyone to give some guideline?