Ask Your Question

Bug or not bug? knnMatch from BruteForceMatcher_GPU

asked 2013-04-23 06:44:07 -0600

updated 2013-04-23 08:40:09 -0600

There is a problem of the match decriptors on the GPU.

OpenCV 2.4.5, CUDA 5.0

I tried to transfer my SURF matcher from the CPU to the GPU and got such a strange result. I use knnMatch and findHomography + perspectiveTransform together with my function, which checks the corners of the bounding box for the result to more precision.

GPU part:

const int baseImagesSize = baseImages.size();
SURF_GPU surf(1500);
surf.extended = false; 

GpuMat keypoints_test_GPU, descriptors_test_GPU;
surf(frame, GpuMat(), keypoints_test_GPU, descriptors_test_GPU);
vector<float> descriptors_test_CPU;
surf.downloadDescriptors(descriptors_test_GPU, descriptors_test_CPU);
Mat descriptors_test_CPU_Mat(descriptors_test_CPU);
vector<Point2f> objs_corners(4);
BruteForceMatcher_GPU< L2<float> > matcher;

vector<KeyPoint> keypoints_test_CPU;
surf.downloadKeypoints(keypoints_test_GPU, keypoints_test_CPU);

for (int i = 0; i < baseImagesSize; ++i)
    //Get the corners from the object
    objs_corners[0] = cvPoint(0,0);
    objs_corners[1] = cvPoint( baseImages[i].cols, 0 );
    objs_corners[2] = cvPoint( baseImages[i].cols, baseImages[i].rows );
    objs_corners[3] = cvPoint( 0, baseImages[i].rows );

    //cout<<endl<<objs_corners[0]<<" "<<objs_corners[1]<<" "<<objs_corners[2]<<" "<<objs_corners[3]<<endl;
    GpuMat keypoints_tmp_GPU, descriptors_tmp_GPU;
    surf(baseImages[i], GpuMat(), keypoints_tmp_GPU, descriptors_tmp_GPU);
    GpuMat trainIdx, distance;

    vector< vector<DMatch> > matches;
    matcher.knnMatch(descriptors_test_GPU, descriptors_tmp_GPU, matches, 2);
    vector<KeyPoint> keypoints_tmp_CPU;

    surf.downloadKeypoints(keypoints_tmp_GPU, keypoints_tmp_CPU);
    std::vector<DMatch > good_matches;

    for(int k = 0; k < min(descriptors_test_CPU_Mat.rows-1,(int) matches.size()); k++) //THIS LOOP IS SENSITIVE TO SEGFAULTS
        if((matches[k][0].distance < 0.6*(matches[k][1].distance)) && ((int) matches[k].size()<=2 && (int) matches[k].size()>0))

    vector<Point2f> obj;
    vector<Point2f> scene;
    vector<Point2f> scene_corners(4);
    Mat H;
    Mat img (baseImages[i]), img_matches, frame_cpu (frame);
    std::ostringstream o_stream;
        drawMatches( img, keypoints_tmp_CPU, frame_cpu, keypoints_test_CPU, good_matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
        cout<<"Error in drawMatches name: "<< baseImagesNames[i]<<endl;

    if (good_matches.size() >= 4)
        for( int k = 0; k < good_matches.size(); k++ )
            //Get the keypoints from the good matches
            obj.push_back( (keypoints_tmp_CPU)[ good_matches[k].queryIdx ].pt );
            scene.push_back( keypoints_test_CPU[ good_matches[k].trainIdx ].pt );

        cout<<good_matches.size()<<" "<<baseImagesNames[i]<<endl;
        H = findHomography( obj, scene, CV_RANSAC);
        perspectiveTransform( objs_corners, scene_corners, H);
        bool falseDetect = isSmallAngle(scene_corners);
        //cout<< falseDetect<< endl;
            cout<<"DETECT "<<baseImagesNames[i]<<endl;



Bad result on GPU (MIN_HESSIAN==1500): surf(1500)

Bad result on GPU (MIN_HESSIAN==400): surf(400)

CPU part:

SurfFeatureDetector detector( MIN_HESSIAN );//MIN_HESSIAN==400
const int baseImagesSize = baseImages.size();
vector< vector<KeyPoint> > kp_objects(baseImagesSize);

//Calculate descriptors (feature vectors)
SurfDescriptorExtractor extractor;
vector<Mat> des_objects(baseImagesSize);
FlannBasedMatcher matcher;
//namedWindow("SURF feature detector");
vector< vector<Point2f> > objs_corners(baseImagesSize,vector<Point2f>(4));

for (int i = 0; i < baseImagesSize; ++i)
    detector.detect(baseImages[i], kp_objects[i]);
    extractor.compute(baseImages[i], kp_objects[i], des_objects[i]);

    //Get the corners from the object
    (objs_corners[i])[0] = cvPoint(0,0);
    (objs_corners[i])[1] = cvPoint( baseImages[i].cols, 0 );
    (objs_corners[i])[2] = cvPoint( baseImages[i].cols, baseImages[i].rows );
    (objs_corners[i])[3] = cvPoint( 0, baseImages[i].rows );

Mat des_image;
std::vector<KeyPoint> kp_image;
Mat image;
cvtColor(frame, image, CV_RGB2GRAY);
detector.detect( image, kp_image );
extractor.compute( image, kp_image, des_image );

for (int i = 0; i < baseImagesSize; ++i)
    Mat img_matches;
    std ...
edit retag flag offensive close merge delete



Maybe there exist an option, which is differently set, like with the ORB descriptor: see e.g. . One note: could you please update this question with the content of the stackoverflow-question, so that it these posts are independent - thank you!

Guanta gravatar imageGuanta ( 2013-04-23 07:23:34 -0600 )edit

Hmm.. I'll try it. How can I download descriptors from ORB_GPU? In my code I used surf.downloadDescriptors(descriptors_test_GPU, descriptors_test_CPU)

Grigory Velmozhin gravatar imageGrigory Velmozhin ( 2013-04-23 08:01:18 -0600 )edit

I just meant, that the bug may be similar to the one in the link, i.e. that maybe an option is not set in the GPU version but is set in the CPU. However, Vladislav already figured your concrete problem out ;) .

Guanta gravatar imageGuanta ( 2013-04-23 08:08:47 -0600 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2013-04-23 08:07:54 -0600

Vladislav Vinogradov gravatar image

The problem is in drawMatches call.

You match descriptors_test_GPU -> descriptors_tmp_GPU:

matcher.knnMatch(descriptors_test_GPU, descriptors_tmp_GPU, matches, 2);

But call drawMatches with keypoints_tmp_CPU -> keypoints_test_CPU:

 drawMatches(img, keypoints_tmp_CPU, frame_cpu, keypoints_test_CPU, ...);

Swap parameters in drawMatches call:

 drawMatches(frame_cpu, keypoints_test_CPU, img, keypoints_tmp_CPU, ...);
edit flag offensive delete link more


Thank you for answer! It`s cool! But this check did not work

H = findHomography( obj, scene, CV_RANSAC); perspectiveTransform( objs_corners, scene_corners, H); bool falseDetect = isSmallAngle(scene_corners); if(!falseDetect) { cout&lt;&lt;"DETECT "&lt;&lt;baseImagesNames[i]&lt;&lt;endl; }

My function gives a result for the corners of the result border:

9.68458 7.70105 6.36017 4.37667 (in degrees) if I set MIN_HESSIAN to 400 (241 matches) and 90 90 90 90 if I set MIN_HESSIAN to 1500 (25 matches) But on the CPU with MIN_HESSIAN 400, all works fine

Grigory Velmozhin gravatar imageGrigory Velmozhin ( 2013-04-23 08:29:34 -0600 )edit

It is the same error. Swap keypoints_tmp_CPU and keypoints_test_CPU:

obj.push_back( keypoints_test_CPU[ good_matches[k].trainIdx ].pt );

scene.push_back( (keypoints_tmp_CPU)[ good_matches[k].queryIdx ].pt );

Vladislav Vinogradov gravatar imageVladislav Vinogradov ( 2013-04-23 11:04:59 -0600 )edit

Thank you for help! It work!

Grigory Velmozhin gravatar imageGrigory Velmozhin ( 2013-04-24 06:01:53 -0600 )edit

Question Tools

1 follower


Asked: 2013-04-23 06:44:07 -0600

Seen: 1,639 times

Last updated: Apr 23 '13