ORB features are mismatched
I am using ORB feature detection and description of OpenCV 3.2.0 library. Using BRUTEFORCE_HAMMINGLUT I match the detected features of my reference and test image.
I drew lines to see how my matched features are aligning and notice a lot of the detected features are being mismatched. So the mismatching of the features means that my homography estimation calculations are wrong because the system is not able to track the translation and rotation vectors of the subject from reference image and the test image.
I am attaching an example of the mismatching. I have the reference image on the left and the test image on the right half of the picture:
As you can see, some features of the eyes from the reference image are being mismatched with the features of ear and the beard in the test image. If all features were paired correctly between the two images, the lines drawn to connect the paired features would never cross over.
I have tried modifying the settings for ORB to change the sensitivity of the detection but still see the mismatching occur. Any suggestions on how to improve the matching of the features?
Bad matches will always occur. To mitigate that you have to use a robust homography estimation method (e.g. RANSAC-based). Also, see the Lowe's ratio test to try to eliminate most of the bad matches.
See this tutorial for more information about the Lowe's ratio test (ignore the FLANN part). And this tutorial for robust homography estimation based on the RANSAC approach.