Ask Your Question

How to find specified number keypoints in mask

asked 2017-11-20 22:03:48 -0500

yode gravatar image

updated 2017-11-21 01:09:04 -0500

Cross post here

ORB is very fast, but it always too few points,which make me awkward. So I have made a such mask

I want to find 10 keypoints in every white rectangle by ORB. But as the documentation here. The first argument just mean:

The maximum number of features to retain.

This is my current method

//OpenCV 3.3.1 project
using namespace std;
using namespace cv;

Mat highlight(Mat, Mat);
int main() {
    Mat emptyImg = imread("test.jpg", 0);
    Mat test = emptyImg.clone();

    Mat mask(5, 5, CV_8UC1, Scalar(0));
    for (int i = 0; i < mask.rows; i++) {
        uchar* data = mask.ptr<uchar>(i);
        for (int j = 0; j < mask.cols; j++)
            if ((i + j) % 2 == 0)
                data[j] = 255;
    resize(mask, mask, emptyImg.size(), 0, 0, INTER_NEAREST);
    Mat labels, stats, centroids;
    int nmask = connectedComponentsWithStats(mask, labels, stats, centroids, 4, CV_16U);
    stats.convertTo(stats, CV_8U);

    vector<Mat> rects;
    for (int i = 1; i < nmask; i++) {
        uchar* data = stats.ptr<uchar>(i);
        Mat temImg = Mat(mask.size(), CV_8UC1, Scalar(0));
        rectangle(temImg, Rect(data[0], data[1], data[2], data[3]), Scalar(255), FILLED);

    Ptr<ORB> detector = ORB::create(10);
    vector<KeyPoint> keypoints_empty;

    for (int i = 0; i < nmask - 1; i++) {
        vector<KeyPoint> tem_keypoints_empty;
        detector->detect(emptyImg, tem_keypoints_empty,rects[i]);
        //string name="rects";
        //imshow((name + to_string(i)).c_str(), rects[i]);
        keypoints_empty.insert(keypoints_empty.end(), tem_keypoints_empty.begin(), tem_keypoints_empty.end());
    Mat showImg=highlight(emptyImg,mask),resultImg;
    drawKeypoints(showImg, keypoints_empty, resultImg,Scalar(0,255,255));
    imshow("keyImg", resultImg);
    return 0;

Mat highlight(Mat srcImg, Mat mask) {
    if (srcImg.size != mask.size) {
        cout << "Your two images tried to highlight have different SIZE.\n";
    if (mask.channels() != 1)
        cvtColor(mask, mask, COLOR_BGR2GRAY);
    if (mask.type() != CV_8UC1)
        mask.convertTo(mask, CV_8UC1);
    threshold(mask, mask, 0, 255, THRESH_BINARY_INV + THRESH_OTSU);
    cvtColor(mask, mask, COLOR_GRAY2BGR);
    if (srcImg.channels() != 3)
        cvtColor(srcImg, srcImg, COLOR_GRAY2BGR);
    if (srcImg.type() != CV_8UC3)
        srcImg.convertTo(srcImg, CV_8UC3);
    dilate(mask - Scalar(0, 0, 255), mask, Mat(), Point(-1, -1), 2);
    return srcImg - mask;

I don't think any problem in my code, but the resultImg make me confused very much..And this is my test image

As we see, there are two problem in my method.

  1. There are some keypoints out of the white rectange
  2. Some rectangle have a few keypoints even have none

Can anybody can help me out?

edit retag flag offensive close merge delete


i think, you misunderstood something:

Ptr<ORB> detector = ORB::create(10);

this will restrict it to a maximum of 10 kp. while you probably expected it to deliver at least 10.

Some rectangle have a few keypoints even have none

if there's no text, or lines, or corners, there can be no keypoints found.

then, you fooled yourself with that "masking". you draw a checkerboard into your mask, but never use it for kp detection. the rects used in the detection are all white !

berak gravatar imageberak ( 2017-11-21 00:59:28 -0500 )edit

@berak I know that meaning. And I know the flaw in my current method. So I post such question for help to improve, because I hope those white area always have 10 keypoints.

yode gravatar imageyode ( 2017-11-21 01:01:58 -0500 )edit

well, not like this. imho, your whole thing is broken beyond repair. throw it away.

because I hope those white area always have 10 keypoints.

that sounds like a broken expectation already. look at the bottom left / right rectangles, they're simply blank. you can't get any kp there

berak gravatar imageberak ( 2017-11-21 01:06:21 -0500 )edit

I have used that mask(rects[i]) actually. Note here please.

yode gravatar imageyode ( 2017-11-21 01:07:57 -0500 )edit

@berak Since you think my solution is bad. Could you give better thinking? Help..

yode gravatar imageyode ( 2017-11-21 02:41:32 -0500 )edit

ah, right. apologies for getting a bit grumpy/impatient there..

berak gravatar imageberak ( 2017-11-21 03:14:30 -0500 )edit

@berak It can be understood.. much less I always receive your help.

yode gravatar imageyode ( 2017-11-21 03:19:02 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2017-11-21 02:57:47 -0500

berak gravatar image

updated 2017-11-21 09:37:33 -0500

the most important part here is to simplify it. no connectedComponentsWithStats needed here ! also you probably should not iterate over small rois, because orb kp work on 32 pixel patches, thus you have to spare a 16 pixel border per rectangle, which will decrease the already sparse kp count significantly.

just draw your mask properly, and apply to the detection.

(if you want to retain only 10 kp per square, better filter for that later, but again, there cannot be any guarantee, that it will find at least 10)

// Mat ocv = ... // the input (bgr)
Mat gray; cvtColor(ocv,gray,COLOR_BGR2GRAY);
Mat mask(ocv.size(), CV_8U, Scalar(0));

int w = ocv.cols/5;
int h = ocv.cols/5;
int x=0, y=0;
for (int i=0; i<5; i++) {
    for (int j=0; j<5; j++) {
         int col = (i+j)%2==0 ? 255:0;
    y += h; x=0;

Ptr<ORB> detector = ORB::create();
vector<KeyPoint> keypoints;
detector->detect(gray, keypoints, mask);

Mat chan[3] = {mask, gray,gray};
merge(chan, 3, ocv);
drawKeypoints(ocv, keypoints, ocv,Scalar(0,0,255));

image description

as you can see, it properly respects the mask now in the kp detection.

edit flag offensive delete link more


You this answer can explain why there are some keypoints out of the white rectange in my solution when I iterate over those small rois??

yode gravatar imageyode ( 2017-11-22 03:01:13 -0500 )edit

yes: you get the outline of your rects (from the connected components), then fill them ALL with white, irrespective of the component label. so all your masks are "on".

berak gravatar imageberak ( 2017-11-22 03:08:35 -0500 )edit

Thanks any way, I think I have find the real reason here

yode gravatar imageyode ( 2017-11-22 04:06:03 -0500 )edit

Question Tools

1 follower


Asked: 2017-11-20 22:03:48 -0500

Seen: 962 times

Last updated: Nov 21 '17