Revision history [back]

I'm moving this to an answer so I can write longer.

Here is the setup you should be using. With the "real" chessboard being flatter than that one, I just grabbed it.

The reason you need to do it this way is because when you find the location of the chessboard, it finds the position of the top left corner of the chessboard and the orientation of the plane of the chessboard. Since you are trying to find the top left corner of the screen and the plane of the screen, you need them to match up. So you put the screen chessboard flat on the screen with it's corner in the corner of the screen.

So you have two cameras and two chessboards. You should take a picture from both cameras with both chessboards in the same spot. For better results, move the secondary camera (the one taking this picture) around and average the results.

From these pictures you will get three transformations.

1. From primary camera to real chessboard.
2. From secondary camera to real chessboard.
3. From secondary camera to screen chessboard.

What you want to find is from primary camera to screen chessboard. So you need to reverse the transformation from secondary camera to real chessboard, and now you have the three you need. primary->real->secondary->screen. And from that, obviously, you can get the inverse, if you need it.

I'm moving this to an answer so I can write longer.

Here is the setup you should be using. With the "real" chessboard being flatter than that one, I just grabbed it. it. Oh, and the picture on the screen should be shown in full screen mode so the corner of the board and screen match.

The reason you need to do it this way is because when you find the location of the chessboard, it finds the position of the top left corner of the chessboard and the orientation of the plane of the chessboard. Since you are trying to find the top left corner of the screen and the plane of the screen, you need them to match up. So you put the screen chessboard flat on the screen with it's corner in the corner of the screen.

So you have two cameras and two chessboards. You should take a picture from both cameras with both chessboards in the same spot. For better results, move the secondary camera (the one taking this picture) around and average the results.

From these pictures you will get three transformations.

1. From primary camera to real chessboard.
2. From secondary camera to real chessboard.
3. From secondary camera to screen chessboard.

What you want to find is from primary camera to screen chessboard. So you need to reverse the transformation from secondary camera to real chessboard, and now you have the three you need. primary->real->secondary->screen. And from that, obviously, you can get the inverse, if you need it.

I'm moving this to an answer so I can write longer.

Here is the setup you should be using. With the "real" chessboard being flatter than that one, I just grabbed it. Oh, and the picture on the screen should be shown in full screen mode so the corner of the board and screen match.

The reason you need to do it this way is because when you find the location of the chessboard, it finds the position of the top left corner of the chessboard and the orientation of the plane of the chessboard. Since you are trying to find the top left corner of the screen and the plane of the screen, you need them to match up. So you put the screen chessboard flat on the screen with it's corner in the corner of the screen.

So you have two cameras and two chessboards. You should take a picture from both cameras with both chessboards in the same spot. For better results, move the secondary camera (the one taking this picture) around and average the results.

From these pictures you will get three transformations.

1. From primary camera to real chessboard.
2. From secondary camera to real chessboard.
3. From secondary camera to screen chessboard.

What you want to find is from primary camera to screen chessboard. So you need to reverse the transformation from secondary camera to real chessboard, and now you have the three you need. primary->real->secondary->screen. And from that, obviously, you can get the inverse, if you need it.

//Secondary Camera to Space Chessboard
<Snip Calculation>
Rodrigues(rvecs[0], R2);
tvecs[0].copyTo(T2);

//Secondary Camera to Screen Chessboard
Rodrigues(rvecs[0], R3);
R3 = R3.t();
T3 = -R3*tvecs[0];

//Primary Camera to Space Chessboard
flip(image, image, 1);
Rodrigues(rvecs[0], R1);
R1 = R1.t();
T1 = -R1*tvecs[0];

Mat translation(3, 1, CV_64F);
translation.setTo(0);
translation = R3*translation + T3;
translation = R2*translation + T2;
translation = R1*translation + T1;

[-96.78414539354907;
 -471.6594258771706;
 16.61072172695242]