reframing an image using a 3D Transform - need help

edit

Hi, I am detecting a square pattern in an image and retrieving its pose using SolvePnP(), which gives me a translation vector in pixel unit and a rotation vector. I would now like to transform my source image using this translation and rotation so that I can display only the sub-part containing the pattern "flatten out" in 2D. The result would be a square image of the pattern as it is in the source image but re-alligned in 2D. I would obtain this by using warPerspective().

I tried getPerspective() + warPerspective instead and it works partially well: the sub-part is indeed retrieved but since this gets me a 2D transform and the pattern in the scene is rotated in 3D, it does not compensate properly and I get a square image of a plane at an angle.

I looked at this: http://jepsonsblog.blogspot.com/2012/11/rotation-in-3d-using-opencvs.html to build the perspective transform out of the 3D translation and rotation. I get confusing results in the final image transform when building the image transformation. If I skip the translation and rotation component, using only a transformation matrix (trans) composed of:

//------
// Projection 2D -> 3D
matrix Mat A1 = (Mat_<double>(4,3) <<
1, 0, -w/2,
0, 1, -h/2,
0, 0, 0,
0, 0, 1);

// 3D -> 2D
matrix Mat A2 =
(Mat_<double>(3,4) <<
f, 0, w/2, 0,
0, f, h/2, 0,
0, 0, 1, 0);

Mat trans = A2 * A1;

// warp perspective according to trans... The result is wrong..
warpPerspective(input, output, trans, input.size(), INTER_LANCZOS4);
//------

I get a black image. To give you an idea of actual values given my calibration:

camera Matrix:
[954.81 | 0 | 306.26
0 | 923.77 | 236.55
0 | 0 | 1]

A2:
[954.81 | 0 | 306.26 | 0
0 | 923.77 | 236.55 | 0
0 | 0 | 1 | 0]

A1:
[1 | 0 | -306.26
0 | 1 | -236.55
0 | 0 | 0
0 | 0 | 1]

final perpective image transform (trans):
(954.81 | 0 | -292420.
0 | 923.77 | -218518.
0 | 0 | 0)

there is clearly something wrong with the last column of this transformation matrix, can you point me out what it is, I don't understand? Thank you very much,