I am calculating the fundamental matrix between two successive frames of a kinect, and according to Hartley and Zisserman the matrix should satisfy x'Fx=0.
I wrote up some quick code that I thought might do the job, but it's not working perfectly, which I will detail at the end of this question:
Mat RansakMask;
Mat F = findFundamentalMat(good_matches1,good_matches2,8,3.0,0.99,RansakMask);
cout << "Number of matches: " << RansakMask.size().height << endl;
int matches[2];
matches[0]=0;matches[1]=0;
for (int j = 0; j < (good_matches1.size()); j++)
{
if(RansakMask.at<uchar>(j, 0) != 1)continue;
cv::Mat p1(3,1, CV_64FC1), p2(3,1, CV_64FC1);
p1.at<double>(0) = good_matches1.at(j).x;
p1.at<double>(1) = good_matches1.at(j).y;
p1.at<double>(2) = 1.0;
p2.at<double>(0) = good_matches2.at(j).x;
p2.at<double>(1) = good_matches2.at(j).y;
p2.at<double>(2) = 1.0;
Mat m = (p1.t()*F*p2);
int i = (abs(m.at<double>(0))<0.5)?0:1;
matches[i]++;
}
cout << "Number of correct: " << matches[0] << endl << "Number of wrong: " << matches[1] << endl;
if(matches[1]>matches[0])
{
cout << "Fundamental Mat is wrong" << endl;
}
The basic idea was, out of all the inliers, compute which were accurate (<0.5f), and which weren't. If there are more accurate than wrong, we can proceed assuming F is relatively accurate.
However, when holding the camera still, this seems to work 9 frames out of 10, on the tenth frame usually giving an answer like (2 correct, 198 wrong) whereas every other frame is typically (198 correct, 2 wrong) etc.
When I move the camera, this turns into more of a 50% chance to be wrong. I don't want to have to throw away F every second frame, so I feel I must be missing something with this code.
Any advice would be appreciated. Thanks.
