Feature recognition: perspective transform
I have implemented the C++ code found here in Java. However, it does not seem to work as I am getting this instead of a nice green outline around the detected image. Can anyone point out where the problem is?
My code is as following (and it's not the most beautiful one because of the so many needed conversions):
Note I am doing this matching with multiple images and the best matching (and other things) is stored at index 'bestMatch'; so in the scene image (index 0), I am looking for the best image match stored at index 1 to n, with bestMatch deemed best based on shortest distance.
//get the best match and show the matching with the original picture
Mat matOut = new Mat();
MatOfDMatch finalMatchess = new MatOfDMatch();
DMatch[] finalMatches = new DMatch[goodMatches[bestMatch].size()]; //goodMatches contains the matches with 3*min_dist as in the C++ code
for (int x=0;x<goodMatches[bestMatch].size();x++) {
finalMatches[x] = goodMatches[bestMatch].get(x);
}
finalMatchess.fromArray(finalMatches);
Features2d.drawMatches(mats[bestMatch],keyPoints[bestMatch],mats[0],keyPoints[0],finalMatchess,matOut);
// stored the indexes for the good matches
int[] usedQueryIds = new int[goodMatches[bestMatch].size()];
int[] usedTrainIds = new int[goodMatches[bestMatch].size()];
for (int x=0;x<goodMatches[bestMatch].size();x++) {
usedQueryIds[x] = goodMatches[bestMatch].get(x).queryIdx;
usedTrainIds[x] = goodMatches[bestMatch].get(x).trainIdx;
}
//get the points used in the matching
Point[] kpBox = new Point[goodMatches[bestMatch].size()];
Point[] kpBox2 = new Point[goodMatches[bestMatch].size()];
KeyPoint[] kpp = keyPoints[bestMatch].toArray();
for (int q=0;q<kpBox.length;q++) {
kpBox[q] = kpp[usedQueryIds[q]].pt;
kpBox2[q] = kp[usedTrainIds[q]].pt;
}
MatOfPoint2f bla = new MatOfPoint2f();
MatOfPoint2f bla2 = new MatOfPoint2f();
bla.fromArray(kpBox);
bla2.fromArray(kpBox2);
Mat H = Calib3d.findHomography(bla,bla2,Calib3d.RANSAC,5.);
//create the points at the corners
List<Point> corners = new ArrayList<Point>();
corners.add(new Point(0,0));
corners.add(new Point(mats[bestMatch].cols(),0));
corners.add(new Point(mats[bestMatch].cols(),mats[bestMatch].rows()));
corners.add(new Point(0,mats[bestMatch].rows()));
List<Point> sceneCorners = new ArrayList<Point>();
//conversions
MatOfPoint2f cornersMat = new MatOfPoint2f();
cornersMat.fromList(corners);
MatOfPoint2f sceneCornersMat = new MatOfPoint2f();
sceneCornersMat.fromList(sceneCorners);
Core.perspectiveTransform(cornersMat,sceneCornersMat,H);
Point[] list = sceneCornersMat.toArray();
Log.d(TAG,list[0].x + ",," + list[0].y); //HERE they are too close together, not even forming a square
Log.d(TAG,list[1].x + ",," + list[1].y);
Log.d(TAG,list[2].x + ",," + list[2].y);
Log.d(TAG,list[3].x + ",," + list[3].y);
sceneCornersMat.put(0,0, sceneCornersMat.get(0,0)[0] + mats[bestMatch].cols(),sceneCornersMat.get(0,0)[1]);
sceneCornersMat.put(1,0, sceneCornersMat.get(1,0)[0] + mats[bestMatch].cols(),sceneCornersMat.get(1,0)[1]);
sceneCornersMat.put(2,0, sceneCornersMat.get(2,0)[0] + mats[bestMatch].cols(),sceneCornersMat.get(2,0)[1]);
sceneCornersMat.put(3,0, sceneCornersMat.get(3,0)[0] + mats[bestMatch].cols(),sceneCornersMat.get(3,0)[1]);
Imgproc.line(matOut,new Point(sceneCornersMat.get(0,0)),new Point(sceneCornersMat.get(1,0)), new Scalar(0,255 ...
You have too few matches and you have at least 2 wrong matches (at least 4 points are needed for the computation of the homography). Also, binary images are not good for features as they need texture information. You could try to increase your threshold or try ratio distance threshold (used in this tutorial) or try other types of detectors / descriptors.
@Eduardo Thank you! I played around with the threshold and indeed, I got it to show a square, so the code is correct; it is obvious though that it's now 'overfitting' just for this particular image. I will try the ratio distance and perhaps other techniques I have found in a paper by Lowe, such as distance between closest neighbor to 2nd closest neighbor and/or clustering with Hough transform.