Rotation and Translation question

edit

I'm not sure that I understand 100% what you're trying to do, but here goes.

(r,t) in this case represent a rigid transform that convert points from the chessboard coordinate system to the camera coordinate system (I don't know the convention for which one is the world coordinate system). You're right about needing the Rodrigues transform, but you have the multiplication order wrong. Say you use Rodrigues to get a rotation matrix R, and you have a point X=[x y z]^T in the chessboard coordinate system (X must be a column-vector). Then to express the point in the world coordinate system, you want R*X + t.

The approach of making the 4x4 matrix should also work. This is based on the concept of homogeneous coordinates (might help with Googling). You should be able to just invert that to get the inverse transformation. Again, the order matters. If P is the 4x4 matrix, it should be P*[0 0 0 1]^T(or whatever the vector is, and again, it must be a column vector).

You should also be able to obtain the inverse without homogeneous coordinates. Let's denote it (R',t'). I'm pretty sure R' = R^T (a rotation matrix is orthonormal, so its inverse is just the transpose). And t' = -R^T * t. You could even take both approaches and check that they give basically the same inverse.