Revision history [back]

As I understand your problem, you have:

one camera, with an AR marker affixed on it, looking to a chessboard
for each chessboard corner, you have the 2D image coordinate and the 3D object coordinate with respect to the AR marker frame
you want to know the transformation between the camera frame and the marker frame

Setup1

Unless you know exactly the 3D coordinate of each chessboard corner with respect to the AR marker frame and unless the chessboard doesn't move, I don't see why it doesn't work.

If one of the two previous conditions is false, the setup should be in my opinion:

one camera, with an AR marker affixed on it, looking to a chessboard
another camera looking at the chessboard and the marker in the same time

Setup2

To obtain the transformation between the first camera and the marker: Equation

Anyway, a figure could help you if I did not understand your problem.

(Note this answer was made before you add all your edit remarks.)

I think that a figure with all the transformation matrix and with all the frames could really help (you and/or us).