Ask Your Question
4

how do descriptors treat image bounds

asked 2013-08-07 11:07:32 -0600

christoph gravatar image

Given that a keypoint is located near an image edge (e.g. at Image(2, 2) ). What does the descriptor do if it describes a region which has a 10 px radius?

Does it compute the values for valid Pixels only? Or does it get rejected completely?

edit retag flag offensive close merge delete

Comments

1

Afaik all implemented descriptors do it as Steven already explained, the regions around the border are just ignored (the possible associated keypoints are deleted as well), no feature will be computed for them.

Guanta gravatar imageGuanta ( 2013-08-08 06:48:19 -0600 )edit
1

That's what I thought. I'm trying to implement the LEHF descriptor, which describes not points but lines. See this paper: http://www.bmva.org/bmvc/2012/BMVC/paper083/paper083.pdf They don't mention how they handle this. Discarding the whole line might not be the best option, because there will allways be some continuous lines reaching over the whole image eg. from left to right. Do you have any thoughts on this?

christoph gravatar imagechristoph ( 2013-08-09 04:40:46 -0600 )edit
1

btw: If anyone of you might care to write an answer, I'll gladly accept it as the right answer.

christoph gravatar imagechristoph ( 2013-08-09 04:44:19 -0600 )edit

About the lines, I guess you could check if a found line is close to an existing border, then remove some of the pixels at the beginning and end of it, just to reduce the region of interest?

StevenPuttemans gravatar imageStevenPuttemans ( 2013-08-09 05:11:08 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
2

answered 2013-08-13 05:38:14 -0600

updated 2013-08-13 05:39:39 -0600

For Brisk, it ignores the keypoint.

//Remove keypoints very close to the border
  size_t ksize = keypoints.size();
  std::vector<int> kscales; // remember the scale per keypoint
  kscales.resize(ksize);
  static const float log2 = 0.693147180559945f;
  static const float lb_scalerange = (float)(log(scalerange_) / (log2));
  std::vector<cv::KeyPoint>::iterator beginning = keypoints.begin();
  std::vector<int>::iterator beginningkscales = kscales.begin();
  static const float basicSize06 = basicSize_ * 0.6f;
  for (size_t k = 0; k < ksize; k++)
  {
    unsigned int scale;
      scale = std::max((int) (scales_ / lb_scalerange * (log(keypoints[k].size / (basicSize06)) / log2) + 0.5), 0);
      // saturate
      if (scale >= scales_)
        scale = scales_ - 1;
      kscales[k] = scale;
    const int border = sizeList_[scale];
    const int border_x = image.cols - border;
    const int border_y = image.rows - border;
    if (RoiPredicate((float)border, (float)border, (float)border_x, (float)border_y, keypoints[k]))
    {
      keypoints.erase(beginning + k);
      kscales.erase(beginningkscales + k);
      if (k == 0)
      {
        beginning = keypoints.begin();
        beginningkscales = kscales.begin();
      }
      ksize--;
      k--;
    }
  }

Also for freak:

 // compute the scale index corresponding to the keypoint size and remove keypoints close to the border
    if( scaleNormalized ) {
        for( size_t k = keypoints.size(); k--; ) {
            //Is k non-zero? If so, decrement it and continue"
            kpScaleIdx[k] = max( (int)(log(keypoints[k].size/FREAK_SMALLEST_KP_SIZE)*sizeCst+0.5) ,0);
            if( kpScaleIdx[k] >= FREAK_NB_SCALES )
                kpScaleIdx[k] = FREAK_NB_SCALES-1;

            if( keypoints[k].pt.x <= patternSizes[kpScaleIdx[k]] || //check if the description at this specific position and scale fits inside the image
                 keypoints[k].pt.y <= patternSizes[kpScaleIdx[k]] ||
                 keypoints[k].pt.x >= image.cols-patternSizes[kpScaleIdx[k]] ||
                 keypoints[k].pt.y >= image.rows-patternSizes[kpScaleIdx[k]]
               ) {
                keypoints.erase(kpBegin+k);
                kpScaleIdx.erase(ScaleIdxBegin+k);
            }
        }
    }

Also for Brief:

    //Remove keypoints very close to the border

KeyPointsFilter::runByImageBorder(keypoints, image.size(), PATCH_SIZE/2 + KERNEL_SIZE/2);


void KeyPointsFilter::runByImageBorder( vector<KeyPoint>& keypoints, Size imageSize, int borderSize )
{

    if( borderSize > 0)
    {
        if (imageSize.height <= borderSize * 2 || imageSize.width <= borderSize * 2)
            keypoints.clear();
        else
            keypoints.erase( std::remove_if(keypoints.begin(), keypoints.end(),
                                       RoiPredicate(Rect(Point(borderSize, borderSize),
                                                         Point(imageSize.width - borderSize, imageSize.height - borderSize)))),
                             keypoints.end() );
    }
}

Not what exactly is edgeThreshold in Orb, but is seems to ignore the keypoints:

 // Remove keypoints very close tothe border
KeyPointsFilter::runByImageBorder(keypoints, imagePyramid[level].size(), edgeThreshold);
edit flag offensive delete link more

Comments

Not sure about Surf and Sift, wait for someone else to comment.

GilLevi gravatar imageGilLevi ( 2013-08-13 05:44:38 -0600 )edit

I marked this as the right answer, because it is the actual source code from https://github.com/Itseez/opencv/tree/master/modules/features2d/src. So this should be proof enough. Thanks to all who participated in answering my question.

christoph gravatar imagechristoph ( 2013-10-07 05:34:15 -0600 )edit
1

answered 2013-08-09 05:07:42 -0600

updated 2013-08-13 04:13:45 -0600

Actually I think this is a good question. I would guess it will only threat image pixels that stay half the radius from the border, but I am not sure of this. Other possibility is that it floods the 'unknown' pixels with the edge values, but this would lead to very misleading results.

I guess that is the most logical approach indeed ... Else the borders will have a way to high response to cornerness.


EDIT: Since I am still convinced that I am correct, I went into a deeper investigation after reading this topic. Looking into how LBP feature descriptor calculates it values, I found it specific that exactly 1 pixel border is left out. This seems to confirm the fact that there is actually an ignoring of the actual border pixels, in order not to create unmeaning values.

edit flag offensive delete link more

Comments

1

-1 Sorry, to say that, but you are guessing. The OP is not asking for thoughts, opinions or "how would you handle this" but facts.

SR gravatar imageSR ( 2013-08-09 06:56:04 -0600 )edit

I just swapped my 'thougths' to an answer based on the original owners request. Thats why I did it, I never suggested it could be the correct solution. If you prefer it to be back at the comments section, no problem for me.

StevenPuttemans gravatar imageStevenPuttemans ( 2013-08-09 07:48:57 -0600 )edit
2

Maybe I was a bit to hastefull to suggest answering the question like this. It is a good idea to get this "suspicion" confirmed by some one official, although I'm quite positive that Steven is actually right. I'll leave it unmarked for now and wait for confirmation. The reason why I rushed him into answering was that there are so many unanswered questions on this site which lessens the overall quality a bit...

christoph gravatar imagechristoph ( 2013-08-09 08:50:26 -0600 )edit

It is not that I mind this single downvote @christoph, so don't worry. I am also interested in seeing an official confirm my thoughts, might do a search in code next week to confirm my 'suspicion'.

StevenPuttemans gravatar imageStevenPuttemans ( 2013-08-10 14:49:42 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2013-08-07 11:07:32 -0600

Seen: 823 times

Last updated: Aug 13 '13