Stitching Problems with Drone Spectral Imagery (BundleBlock Adj?) [closed]

edit

Hey Team, we are using OpenCV to process Gigs of drone spectral imagery data and have hit a wall creating a mosaic of the data (details below). Looking to get connected with someone who (possibly for hire) can get us to a solution that allows us to perform some kind of bundle block adjustment on the data during the mosaic'ing process.

Thanks in Advance!

Tim Haynie [email protected]

The goal is to stitch a set of unordered images obtained from an aerial platform to create a single image.

Additional Information

The images do contain pitch, roll and yaw information (but can be assumed to be shot nadir) and they also contain geo coordinates. This information gets us close to a solution, but the sensors are not accurate enough to form a good looking result (Some of the individual images appear to be positioned up to 5 yards from their actual position when compared to either each other or to a ground control image. The images are also not rotationally oriented to each other correctly. Again, this varies, but appears to be off by up to 15 degrees in some cases).

Software

The application is written in c++ using OpenCV for the image processing, SBA (sparse bundle adjust) libraries for final BA, and Gdal for conversion from JPEG to geotiff.

The Approach

I have tried many things so far, but this is what seems to get me the closest results:

Load in the camera intrinsic values. Perform feature matching using SURF. Obtain the estimated Homography between images 1 and 2 using Open CV methods, including RANSAC to remove outliers from the matches. Extract the Rotations and Translations from the Homography and project the image using Image Warp functions. Blend the images

As I mentioned, the results are close, but still noticeable errors in the rotation and translation between the images.

I am assuming a bundle adjustment should “fix” the images, but I have not been able to see this functionality work correctly. The OpenCV Bundle Adjust is for rotations only, so I have been researching using another library that should correct for rotation and translations. So far, the results are worse than pre-bundle adjustment, so I am currently trying to figure out this library. The version I have expects a set of 3D object points, and 2 sets of image points. I have been making an assumption the image is planer, and setting the z-values for the object points to 1 (I don’t have a DEM to work from. The image is of a football field and is pretty flat, so I am assuming this assumption should work. However, it is also possible the assumption is incorrect, so figuring out a bundle adjustment for just the 2D points in the image would be acceptable at this time.

Problems

Need to figure out if there is a better way of estimating the initial Homography. Possibly looking at an iterative approach to refine the results. Is there a better method ... (more)