# reframing an image using a 3D Transform - need help

Hi, I am detecting a square pattern in an image and retrieving its pose using SolvePnP(), which gives me a translation vector in pixel unit and a rotation vector. I would now like to transform my source image using this translation and rotation so that I can display only the sub-part containing the pattern "flatten out" in 2D. The result would be a square image of the pattern as it is in the source image but re-alligned in 2D. I would obtain this by using warPerspective().

I tried getPerspective() + warPerspective instead and it works partially well: the sub-part is indeed retrieved but since this gets me a 2D transform and the pattern in the scene is rotated in 3D, it does not compensate properly and I get a square image of a plane at an angle.

I looked at this: http://jepsonsblog.blogspot.com/2012/11/rotation-in-3d-using-opencvs.html to build the perspective transform out of the 3D translation and rotation. I get confusing results in the final image transform when building the image transformation. If I skip the translation and rotation component, using only a transformation matrix (trans) composed of:

//------

// Projection 2D -> 3D

matrix Mat A1 = (Mat_<double>(4,3) <<

1, 0, -w/2,

0, 1, -h/2,

0, 0, 0,

0, 0, 1);// 3D -> 2D

matrix Mat A2 =

(Mat_<double>(3,4) <<

f, 0, w/2, 0,

0, f, h/2, 0,

0, 0, 1, 0);Mat trans = A2 * A1;

// warp perspective according to trans... The result is wrong..

warpPerspective(input, output, trans, input.size(), INTER_LANCZOS4);

//------

I get a black image. To give you an idea of actual values given my calibration:

camera Matrix:

[954.81 | 0 | 306.26

0 | 923.77 | 236.55

0 | 0 | 1]

A2:

[954.81 | 0 | 306.26 | 0

0 | 923.77 | 236.55 | 0

0 | 0 | 1 | 0]

A1:

[1 | 0 | -306.26

0 | 1 | -236.55

0 | 0 | 0

0 | 0 | 1]

final perpective image transform (trans):

(954.81 | 0 | -292420.

0 | 923.77 | -218518.

0 | 0 | 0)

there is clearly something wrong with the last column of this transformation matrix, can you point me out what it is, I don't understand? Thank you very much,