Detecting objects and computing pose of object/camera

edit

Hello,

Assuming there are two cameras in 3D space, spaced apart, looking at the same scene. I am trying to achieve the following through opencv: (please correct me if my approach is wrong)

1. Camera1 which is fixed, looks at an object, computes pose of the object through solvePnP.
2. Camera2's position is noisy, so there's noise in terms of both rotation and translation. It looks at the same object, and computes the pose at every frame.
3. Frame-by-frame, I would then solve for the pose of the moving camera and use that info for stabilizing it.

Is it possible to do this by detecting a planar object in the scene (not like a checkerboard), and using it for pose estimation? Any pointers or suggestions would be helpful.

Thanks,

Sai