Hi, I am detecting a square pattern in an image and retrieving its pose using SolvePnP(), which gives me a translation vector in pixel unit and a rotation vector. I would now like to transform my source image using this translation and rotation so that I can display only the sub-part containing the pattern "flatten out" in 2D. The result would be a square image of the pattern as it is in the source image but re-alligned in 2D. I would obtain this by using warPerspective().
I tried getPerspective() + warPerspective instead and it works partially well: the sub-part is indeed retrieved but since this gets me a 2D transform and the pattern in the scene is rotated in 3D, it does not compensate properly and I get a square image of a plane at an angle.
I looked at this: http://jepsonsblog.blogspot.com/2012/11/rotation-in-3d-using-opencvs.html to build the perspective transform out of the 3D translation and rotation. I get confusing results in the final image transform when building the image transformation. If I skip the translation and rotation component, using only a transformation matrix (trans) composed of:
//------
// Projection 2D -> 3D matrix Mat A1 = (Mat_<double>(4,3) << 1, 0, -w/2, 0, 1, -h/2, 0, 0, 0, 0, 0, 1);// 3D -> 2D matrix Mat A2 = (Mat_<double>(3,4) << f, 0, w/2, 0, 0, f, h/2, 0, 0, 0, 1, 0);
Mat trans = A2 * A1 warpPerspective(input, output, trans, input.size(), INTER_LANCZOS4); //------
I get a black image. To give you an idea of actual values given my calibration:
camera Matrix: [954.81 | 0 | 306.26 0 | 923.77 | 236.55 0 | 0 | 1]
A2: [954.81 | 0 | 306.26 | 0 0 | 923.77 | 236.55 | 0 0 | 0 | 1 | 0]
A1: [1 | 0 | -306.26 0 | 1 | -236.55 0 | 0 | 0 0 | 0 | 1]
final perpective image transform (trans): (954.81 | 0 | -292420. 0 | 923.77 | -218518.
0 | 0 | 0)
there is clearly something woring with the last column of this transformation matrix, can you point me out what it is, I don't understand? Thank you very much,