1 | initial version |
Maybe you can use a calibration target like the one used for stereo calibration (chessboard), move the target around the overlapping areas of the images, detect the chessboard on each pair of overlapping images and use that information to calculate the corresponding homography matrices. This process should be done only once or only when you suspect that the cameras might have moved with respect to each other.
If the cameras are fixed then this homography matrices should remain constant and there is no need to recalculate them by using SURF feature matching on each new trio of images, as it is time consuming and error prone as you already experienced.