How can I benefit from having videos of two corresponding view as I estimate homography of the overlapping part?
I try to stitch two views of the scene by estimating homography from overlapping segment. However, because of the nature of approximation of homography matrix, system gives slightly different results for each run. Then since I have same time videos of both views, I think about to use t0matching frames in time to have more robust estimation of H matrix. The question is, how can I get those multiple frames useful for the robustness of the system ?
I assume that your cameras are not moving (is it possible that they are shaking a bit?) and for consecutive image pairs you get slightly different homography matrices and visibly different stitching results. Am I right?
Actually for each run ,even the pair is same, I get different stitching result because of estimation of H matrix is not very robust despite of RANSAC. I decided to use the frames of the videos to enhance the number of mathing interest points so as to have more points for RANSAC. I believe that RANSAC will be more robust with more precise H estimation. I am asking that is this true or how can I use the nature of video for better stitching.?
RANSAC is by definition random. At least to some extent. How many keypoints have you got? I suppose many, among which many are quite similar. That's why each time you run stitching RANSAC is selecting slightly different set for inliers and outliers, which leads to different homographies. But my thinking can be completely wrong. However if it's not, try to limit the number of keypoints to 20-40 and see if it changes anything. And I would rather look at the stitching effect, than at the matrix itself. Minor changes in some matrix elements can result in a very similar image transformation.