good result or bad result for findHomography

edit

Algebraically, both will give the same results when the matrix is singular (or not). (2) is actually a stronger and probably better measure of quality, though you'd have to verify that the largest eigen-value isn't too small too.

OpenCV supports SVD.

My understanding concerning :

1. "Compute the determinant of the homography, and see if it's too close to zero for comfort."

Compute the determinant of the top left 2x2 homography matrix, and check if it's "too close" to zero for comfort...btw you can also check if it's *too *far from zero because then the invert matrix would have a determinant too close to zero. A determinant of zero would mean the matrix is not inversible, too close to zero would mean *singular (like you see the plane object at 90°, which is almost impossible if you use *good matches).

const double det = H.at<double>(0, 0) * H.at<double>(1, 1) - H.at<double>(1, 0) * H.at<double>(0, 1);

One way to do it would be to specify a positive value, higher than 1, say N, and then:

if((fabs(det)>(double)N) || (fabs(det)<(1.0/(double)N)) return false; // bad homography

And while we are at it...if det<0 the homography is not conserving the orientation (clockwise<->anticlockwise), except if you are watching the object in a mirror...it is certainly not good (plus the sift/surf descriptors are not done to be mirror invariants as far as i know, so you would probably don'thave good maches).

if(det<0) return false; // no mirrors in the scene

/* for reading & reference : [Detecting Planar Homographies in an Image Pair Etienne Vincent and Robert Laganiere School of Information Technology and Engineering University of Ottawa Canada K1N 6N5] */

To my experience, it is hard to choose thresholds (e.g. N) such that outliers are removed, but (bad) correct estimates are kept.

If you estimate the homography for a rectangular template, you can draw a quadrangle given by the corner points of this template. Outliers can be identified from quadrangles with self-overlap or extreme corner angles (e.g. >170° or < 30°).

It shouldn't be too hard to check these conditions, but I do not have a code example right now.