Ask Your Question
5

The homography tutorial in java

asked 2013-03-25 04:57:13 -0600

Sylvain gravatar image

updated 2013-03-25 10:55:20 -0600

EDIT : main problem solved (see in comments) ! But i now have a problem with perspectiveTransform()...

OpenCV Error: Assertion failed (scn + 1 == m.cols && (depth == CV_32F || depth == CV_64F)) in unknown function, file ......\src\opencv\modules\core\src\matmul.cpp, line 1926


Original post

Hi,

i'm trying to redo this tutorial in Java... but it doesn't work. The findHomography function didn't work, and the error message isn't very clear for me :

OpenCV Error: Assertion failed (npoints >= 0 && points2.checkVector(2) == npoints && points1.type() == points2.type()) in unknown function, file ..\..\..\src\opencv\modules\calib3d\src\fundam.cpp, line 1074

I realy hope someone will be able to help me... i'm a novice with OpenCV

Here is my code, sorry it's a bit long :

CODE UPDATED

class FindObject {
  public void run(String pathObject, String pathScene, String pathResult) {

System.out.println("\nRunning FindObject");

Mat img_object = Highgui.imread("D:/workspaceSeirich/HelloCV/".concat(pathObject), 0); //0 = CV_LOAD_IMAGE_GRAYSCALE
Mat img_scene = Highgui.imread("D:/workspaceSeirich/HelloCV/".concat(pathScene), 0);

FeatureDetector detector = FeatureDetector.create(4); //4 = SURF 

MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
MatOfKeyPoint keypoints_scene  = new MatOfKeyPoint();

detector.detect(img_object, keypoints_object);
detector.detect(img_scene, keypoints_scene);

DescriptorExtractor extractor = DescriptorExtractor.create(2); //2 = SURF;

Mat descriptor_object = new Mat();
Mat descriptor_scene = new Mat() ;

extractor.compute(img_object, keypoints_object, descriptor_object);
extractor.compute(img_scene, keypoints_scene, descriptor_scene);

DescriptorMatcher matcher = DescriptorMatcher.create(1); // 1 = FLANNBASED
MatOfDMatch matches = new MatOfDMatch();

matcher.match(descriptor_object, descriptor_scene, matches);
List<DMatch> matchesList = matches.toList();

Double max_dist = 0.0;
Double min_dist = 100.0;

for(int i = 0; i < descriptor_object.rows(); i++){
    Double dist = (double) matchesList.get(i).distance;
    if(dist < min_dist) min_dist = dist;
    if(dist > max_dist) max_dist = dist;
}

System.out.println("-- Max dist : " + max_dist);
System.out.println("-- Min dist : " + min_dist);    

LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
MatOfDMatch gm = new MatOfDMatch();

for(int i = 0; i < descriptor_object.rows(); i++){
    if(matchesList.get(i).distance < 3*min_dist){
        good_matches.addLast(matchesList.get(i));
    }
}

gm.fromList(good_matches);

Mat img_matches = new Mat();
Features2d.drawMatches(
        img_object,
        keypoints_object, 
        img_scene,
        keypoints_scene, 
        gm, 
        img_matches, 
        new Scalar(255,0,0), 
        new Scalar(0,0,255), 
        new MatOfByte(), 
        2);

LinkedList<Point> objList = new LinkedList<Point>();
LinkedList<Point> sceneList = new LinkedList<Point>();

List<KeyPoint> keypoints_objectList = keypoints_object.toList();
List<KeyPoint> keypoints_sceneList = keypoints_scene.toList();

for(int i = 0; i<good_matches.size(); i++){
    objList.addLast(keypoints_objectList.get(good_matches.get(i).queryIdx).pt);
    sceneList.addLast(keypoints_sceneList.get(good_matches.get(i).trainIdx).pt);
}

MatOfPoint2f obj = new MatOfPoint2f();
obj.fromList(objList);

MatOfPoint2f scene = new MatOfPoint2f();
scene.fromList(sceneList);

Mat H = Calib3d.findHomography(obj, scene);

LinkedList<Point> cornerList = new LinkedList<Point>();
cornerList.add(new Point(0,0));
cornerList.add(new Point(img_object.cols(),0));
cornerList.add(new Point(img_object.cols(),img_object.rows()));
cornerList.add(new Point(0,img_object.rows()));

MatOfPoint obj_corners = new MatOfPoint();
obj_corners.fromList(cornerList);

MatOfPoint scene_corners = new MatOfPoint();

//ERROR HERE :
//OpenCV Error: Assertion failed (scn + 1 == m.cols && (depth == CV_32F || depth == CV_64F)) in unknown function, file ..\..\..\src\opencv\modules\core\src\matmul.cpp, line 1926
Core.perspectiveTransform(obj_corners, scene_corners, H);

//Draw the lines... later, when the homography will work
/*
Core.line(img_matches, new Point(), new Point(), new ...
(more)
edit retag flag offensive close merge delete

Comments

what happens, if you don't find any matches ? both obj and scene will be empty

berak gravatar imageberak ( 2013-03-25 05:29:09 -0600 )edit

Good questions... i've never try to detect an object which isn't present in the scene. I think it will find something weird because it always find some matching points.

Sylvain gravatar imageSylvain ( 2013-03-25 05:57:26 -0600 )edit

you need at least 4 points (each) for findhomography, so you should already bail out after matcher.match(), if there's less matches

berak gravatar imageberak ( 2013-03-25 07:48:55 -0600 )edit

I've a lot more, i can see that if i only draw all the "good" matches (whose distance is less than 3*min_dist). Depending on the pictures, i've between 20 and 140 items in my 'good_matches' list (if i do a system.out.printn on it)

Sylvain gravatar imageSylvain ( 2013-03-25 08:24:32 -0600 )edit

I think the problem probably come from those lines

for(int i = 0; i&lt;good_matches.size(); i++){
    //Get the keypoints from the good matches
    obj.push_back(keypoints_object.row(good_matches.get(i).queryIdx));
    scene.push_back(keypoints_scene.row(good_matches.get(i).trainIdx));
}

But i can't find why...

Sylvain gravatar imageSylvain ( 2013-03-25 08:56:51 -0600 )edit
1

oh, right. look again at the tutorial, it has:

obj.push_back( keypoints_object[ good_matches[i].queryIdx ].pt ); // note th pt at the end

as of java, i can only guess:

obj.push_back(keypoints_object.row(good_matches.get(i).queryIdx).pt);

yes, you want a list of points there, not a list of KeyPoints

berak gravatar imageberak ( 2013-03-25 09:23:33 -0600 )edit

Oh yes ! I've totally forgot this details. We're on the good way (hum... i hope so...)

But there is no way to do .pt in java. And i don't find any getPt() function. Any idea ?

Sylvain gravatar imageSylvain ( 2013-03-25 09:31:28 -0600 )edit

aw, sorry, more the c++ guy here, but http://docs.opencv.org/java/ states, KeyPoint has a pt

berak gravatar imageberak ( 2013-03-25 09:51:40 -0600 )edit

Okay, it's was a type problem... there is no way to extract from a Mat the good type. We've to use a list. It was, indeed, a problem of type before using findHomography(). The right way is :

LinkedList<Point> objList = new LinkedList<Point>(); LinkedList<Point> sceneList = new LinkedList<Point>();

List&lt;KeyPoint&gt; keypoints_objectList = keypoints_object.toList();
List&lt;KeyPoint&gt; keypoints_sceneList = keypoints_scene.toList();

for(int i = 0; i<good_matches.size(); i++){ objList.addLast(keypoints_objectList.get(good_matches.get(i).queryIdx).pt); sceneList.addLast(keypoints_sceneList.get(good_matches.get(i).trainIdx).pt); }

MatOfPoint2f obj = new MatOfPoint2f(); obj.fromList(objList);

Same with the scene, and then findHomography()

Sylvain gravatar imageSylvain ( 2013-03-25 10:30:27 -0600 )edit

But i now have a problem some lines after, when i do perspectiveTransform(), i've an other ununderstandable error :

OpenCV Error: Assertion failed (scn + 1 == m.cols && (depth == CV_32F || depth == CV_64F)) in unknown function, file ......\src\opencv\modules\core\src\matmul.cpp, line 1926

Sylvain gravatar imageSylvain ( 2013-03-25 10:34:05 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
5

answered 2013-03-26 03:08:43 -0600

Shataya gravatar image

I did this tutorial in Android... I think you have a type problem again.. :) Here is my solution with "Mat" for the corners:

Mat hg = Calib3d.findHomography(obj, scene);

Mat obj_corners = new Mat(4,1,CvType.CV_32FC2);
Mat scene_corners = new Mat(4,1,CvType.CV_32FC2);

obj_corners.put(0, 0, new double[] {0,0});
obj_corners.put(1, 0, new double[] {mat2.cols(),0});
obj_corners.put(2, 0, new double[] {mat2.cols(),mat2.rows()});
obj_corners.put(3, 0, new double[] {0,mat2.rows()});

Core.perspectiveTransform(obj_corners,scene_corners, hg);

Core.line(mat, new Point(scene_corners.get(0,0)), new Point(scene_corners.get(1,0)), new Scalar(0, 255, 0),4);
Core.line(mat, new Point(scene_corners.get(1,0)), new Point(scene_corners.get(2,0)), new Scalar(0, 255, 0),4);
Core.line(mat, new Point(scene_corners.get(2,0)), new Point(scene_corners.get(3,0)), new Scalar(0, 255, 0),4);
Core.line(mat, new Point(scene_corners.get(3,0)), new Point(scene_corners.get(0,0)), new Scalar(0, 255, 0),4);
edit flag offensive delete link more

Comments

You're awesome ! That's work :) Thank you very much, I was stuck since yesterday !

Sylvain gravatar imageSylvain ( 2013-03-26 03:23:04 -0600 )edit

hi im trying to do the same tutorial in android, is your code on github so i can see it as a reference and if not could you upload it?

asd gravatar imageasd ( 2013-07-09 02:37:48 -0600 )edit

I dont know why, i have used this whole code, but it is giving me error in detector.detct() line, Can you please share your code

Mehul Thakkar gravatar imageMehul Thakkar ( 2014-01-06 22:22:23 -0600 )edit

Hi! Thanks for your code ! How to find the rotated angle of object in scene?

prakash gravatar imageprakash ( 2015-10-30 01:19:39 -0600 )edit
0

answered 2013-03-30 03:19:34 -0600

RBishnoi gravatar image

I have been trying to write this in java but couldn't get correct results...Even after using the above code and changes from comments I am not getting correct results oDetection1.png. Please tell me what mistake I am doing?.

Using the above code and changes,code:

public class FindO { public void run() {

    System.out.println("\nRunning FindObject");
    System.loadLibrary("opencv_java244");


    String object_filename = "D:\\box.png";
    String scene_filename = "D:\\box_in_scene.png";

    Mat img_object = Highgui.imread(object_filename, Highgui.CV_LOAD_IMAGE_GRAYSCALE); // 0 =
                                                            // CV_LOAD_IMAGE_GRAYSCALE
    Mat img_scene = Highgui.imread(scene_filename, Highgui.CV_LOAD_IMAGE_GRAYSCALE);

    FeatureDetector detector = FeatureDetector.create(FeatureDetector.SURF); // 4 = SURF

    MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
    MatOfKeyPoint keypoints_scene = new MatOfKeyPoint();

    detector.detect(img_object, keypoints_object);
    detector.detect(img_scene, keypoints_scene);

    DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.SURF); // 2 =
                                                                    // SURF;

    Mat descriptor_object = new Mat();
    Mat descriptor_scene = new Mat();

    extractor.compute(img_object, keypoints_object, descriptor_object);
    extractor.compute(img_scene, keypoints_scene, descriptor_scene);

    DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.FLANNBASED); // 1 =
                                                                // FLANNBASED
    MatOfDMatch matches = new MatOfDMatch();

    matcher.match(descriptor_object, descriptor_scene, matches);
    List<DMatch> matchesList = matches.toList();

    Double max_dist = 0.0;
    Double min_dist = 100.0;

    for (int i = 0; i < matchesList.size(); i++) {
        Double dist = (double) matchesList.get(i).distance;
        if (dist < min_dist)
            min_dist = dist;
        if (dist > max_dist)
            max_dist = dist;
    }

    System.out.println("-- Max dist : " + max_dist);
    System.out.println("-- Min dist : " + min_dist);

    LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
    MatOfDMatch gm = new MatOfDMatch();

    for (int i = 0; i < matchesList.size(); i++) {
        if (matchesList.get(i).distance < (3 * min_dist)) {
            good_matches.addLast(matchesList.get(i));
        }
    }


    gm.fromList(good_matches);

    Mat img_matches = new Mat();
    Features2d.drawMatches(img_object, keypoints_object, img_scene,
            keypoints_scene, gm, img_matches, new Scalar(255, 0, 0),
            new Scalar(0, 0, 255), new MatOfByte(), 2);

    String filename = "D:\\oDetection.png";

    System.out.println(String.format("Writing %s", filename));
    Highgui.imwrite(filename, img_matches);

    LinkedList<Point> objList = new LinkedList<Point>();
    LinkedList<Point> sceneList = new LinkedList<Point>();

    List<KeyPoint> keypoints_objectList = keypoints_object.toList();
    List<KeyPoint> keypoints_sceneList = keypoints_scene.toList();

    for (int i = 0; i < good_matches.size(); i++) {
        objList.addLast(keypoints_objectList.get(good_matches.get(i).queryIdx).pt);
        sceneList
                .addLast(keypoints_sceneList.get(good_matches.get(i).trainIdx).pt);
    }

    MatOfPoint2f obj = new MatOfPoint2f();
    obj.fromList(objList);

    MatOfPoint2f scene = new MatOfPoint2f();
    scene.fromList(sceneList);

    Mat H = Calib3d.findHomography(obj, scene);





    Mat obj_corners = new Mat(4,1,CvType.CV_32FC2);
    Mat scene_corners = new Mat(4,1,CvType.CV_32FC2);



    obj_corners.put(0, 0, new double[] {0,0});
    obj_corners.put(1, 0, new double[] {img_object.cols(),0});
    obj_corners.put(2, 0, new double[] {img_object.cols(),img_object.rows()});
    obj_corners.put(3, 0, new double[] {0,img_object.rows()});

    Core.perspectiveTransform(obj_corners,scene_corners, H);

    Mat img = Highgui.imread(scene_filename, Highgui.CV_LOAD_IMAGE_COLOR);

    Core.line(img, new Point(scene_corners.get(0,0)), new Point(scene_corners.get(1,0)), new Scalar(0, 255, 0),4);
    Core.line(img, new Point(scene_corners.get(1,0)), new Point(scene_corners.get(2,0)), new Scalar(0, 255, 0),4);
    Core.line(img, new Point(scene_corners.get(2,0)), new Point(scene_corners.get(3,0)), new Scalar(0, 255, 0),4);
    Core.line(img, new Point(scene_corners.get(3,0)), new Point(scene_corners.get(0,0)), new Scalar(0, 255, 0),4);

    filename = "D:\\oDetection1.png ...
(more)
edit flag offensive delete link more

Comments

Hi,

Sorry for my late, i havn't see your answer. You shoul try this way :

Mat hg = Calib3d.findHomography(obj, scene, 8, 10);

//8 for RANSAC //10 for the reprojection error

Hop it will help you !

Sylvain gravatar imageSylvain ( 2013-04-18 02:12:05 -0600 )edit

Hi thanks for a good tutorial I tried the same but in real time camera frame processing, even after replacing with Mat hg = Calib3d.findHomography(obj, scene, 8, 10); also tried using CV_RANSAC instead the result is same as shown in the picture above, I can't figure what's wrong.

swamynathan gravatar imageswamynathan ( 2013-07-25 15:36:52 -0600 )edit

Hi Did you manage to solve this ? I am facing the same problem... Thanks

arpit03goel gravatar imagearpit03goel ( 2013-07-25 16:41:47 -0600 )edit

@Sylvian:

There's a slight mistake

Mat hg = Calib3d.findHomography(obj, scene, 8, 10); //8 for RANSAC //10 for the reprojection error

but if you look here there's no 10 as constant field value for the Calib3d in openCV

and it doesn't work. please clarify

swamynathan gravatar imageswamynathan ( 2013-08-08 18:32:19 -0600 )edit

Question Tools

4 followers

Stats

Asked: 2013-03-25 04:57:13 -0600

Seen: 17,597 times

Last updated: Jun 06 '13