Validating the Funadamental Matrix

edit

I am calculating the fundamental matrix between two successive frames of a kinect, and according to Hartley and Zisserman the matrix should satisfy x'Fx=0.

I wrote up some quick code that I thought might do the job, but it's not working perfectly, which I will detail at the end of this question:

        Mat RansakMask;
        Mat F = findFundamentalMat(good_matches1,good_matches2,8,3.0,0.99,RansakMask);
        cout << "Number of matches: " << RansakMask.size().height << endl;
        int matches[2];
        matches[0]=0;matches[1]=0;
        for (int j = 0; j < (good_matches1.size()); j++)
        {
            if(RansakMask.at<uchar>(j, 0) != 1)continue;
            cv::Mat p1(3,1, CV_64FC1), p2(3,1, CV_64FC1);

            p1.at<double>(0) = good_matches1.at(j).x;
            p1.at<double>(1) = good_matches1.at(j).y;
            p1.at<double>(2) = 1.0;

            p2.at<double>(0) = good_matches2.at(j).x;
            p2.at<double>(1) = good_matches2.at(j).y;
            p2.at<double>(2) = 1.0;

            Mat m = (p1.t()*F*p2);
            int i = (abs(m.at<double>(0))<0.5)?0:1;
            matches[i]++;
        }
        cout << "Number of correct: " << matches[0] << endl << "Number of wrong: " << matches[1] << endl;
        if(matches[1]>matches[0])
        {
            cout << "Fundamental Mat is wrong" << endl;
        }

The basic idea was, out of all the inliers, compute which were accurate (<0.5f), and which weren't. If there are more accurate than wrong, we can proceed assuming F is relatively accurate.

However, when holding the camera still, this seems to work 9 frames out of 10, on the tenth frame usually giving an answer like (2 correct, 198 wrong) whereas every other frame is typically (198 correct, 2 wrong) etc.

When I move the camera, this turns into more of a 50% chance to be wrong. I don't want to have to throw away F every second frame, so I feel I must be missing something with this code.

Any advice would be appreciated. Thanks.