How to find surfance orientation (relative to camera) from single depth image?
Supposing I have access to the image stream of a depth camera, and there is a flat surface (e.g. floor, tabletop, etc) within the camera's FoV at all times, how could one estimate the floor vertical and horizontal orientation (or better yet, the rotation matrix/vector) from the camera perspective?
I have access to the camera matrix, therefore I can select multiple points on the surface and reconstruct their 3D coordinates on the camera frame. But how do I use those coordinates to build a transformation matrix (mostly rotation, translation is irrelevant, I'd just need to orient my reference frame orthogonally with the surface)
My main limitation seems to be that I do not have the correspondent coordinates on the surface points (on object/external reference), therefore can't use cv::estimateAffine3D or cv::findHomography or cv::solvePnP.
I've tried to estimate the plane equation using cv::SVD, but resulting fit doesn't seem to be that precise and I am not aware how could I use the plane equation to find the affine transformation matrix.