Drawing a rectangle around detected object with SIFT [closed]

asked 2020-05-28 04:13:43 -0500

albeh95 gravatar image

Hello everyone, I'm trying to draw a rectangle around a detected object from an image to a scene (that's actually the first frame of a video), by using SIFT features and the homography matrix. First of all to get features and matching I'm using the following code:

cv::Ptr<Feature2D> f2d = xfeatures2d::SIFT::create();
Ptr<DescriptorExtractor> extractor = xfeatures2d::SIFT::create();
//part of the code not relevant...
Ptr<BFMatcher> matcher = BFMatcher::create(NORM_L2);
    vector<DMatch> matches;
    matcher->match(descriptors_object, descriptors_scene, matches);

    //-- Localize the object
    std::vector<Point2f> obj;
    std::vector<Point2f> scene;
    for (size_t i = 0; i < matches.size(); i++)
        //-- Get the keypoints from the matches
    Mat mask;
    Mat H = findHomography(obj, scene, RANSAC, 3.0, mask);

    vector<DMatch> good_matches;
    for (int r = 0; r < matches.size(); r++)
        if ((int)mask.at<uchar>(r, 0) != 0)
    //-- Draw matches
    Mat img_matches;
    drawMatches(img_object, keypoints_object, img_scene, keypoints_scene, good_matches, img_matches, 
     Scalar::all(-1), Scalar::all(-1), std::vector<char>());

I'm getting this result: image description

Now I'd like to draw a rectangle around THAT detected book. I wrote the following code:

  //-- Get the corners from the image_1 ( the object to be "detected" )
    std::vector<Point2f> obj_corners(4);
    obj_corners[0] = Point2f(0, 0);
    obj_corners[1] = Point2f((float)img_object.cols, 0);
    obj_corners[2] = Point2f((float)img_object.cols, (float)img_object.rows);
    obj_corners[3] = Point2f(0, (float)img_object.rows);
    std::vector<Point2f> scene_corners(4);

    perspectiveTransform(obj_corners, scene_corners, H);

    //-- Draw lines between the corners (the mapped object in the scene - image_2 )
    line(img_matches, scene_corners[0] + Point2f(img_object.cols, 0), scene_corners[1] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
    line(img_matches, scene_corners[1] + Point2f(img_object.cols, 0), scene_corners[2] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
    line(img_matches, scene_corners[2] + Point2f(img_object.cols, 0), scene_corners[3] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);
    line(img_matches, scene_corners[3] + Point2f(img_object.cols, 0), scene_corners[0] + Point2f(img_object.cols, 0), Scalar(0, 255, 0), 4);

But I'm getting the same result as before, without the rectangle. Whatìs wrong with the code?

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by albeh95
close date 2020-05-29 05:04:14.453911


What is this? It doesn't matches on template. Did you see Multiple View. This lines and all lines go straight to Hartley.

supra56 gravatar imagesupra56 ( 2020-05-28 04:46:29 -0500 )edit

So the problem is in in the first part of the code right? It actually doens't match correctly features... Do you suggest to change dataset?

albeh95 gravatar imagealbeh95 ( 2020-05-28 04:48:57 -0500 )edit

I merely do in python. By looking @ your code. You don't needed a lines to draw an rectangle. ORB automatically draw a rectangle for you. I will post my link later.

supra56 gravatar imagesupra56 ( 2020-05-28 05:35:18 -0500 )edit

try to print out your projected points, most likely they're off-screen (because of a bad homography)

berak gravatar imageberak ( 2020-05-28 09:34:13 -0500 )edit

I changed the matcheds filtering and now it works: instead of using the homography mask to select only the inliners I choose good matches basing only on the Lowe's ratio test. I guess it's ok

albeh95 gravatar imagealbeh95 ( 2020-05-28 13:05:56 -0500 )edit