Ask Your Question
0

Calculate (ORB) descriptors for arbitrary image points

asked 2020-07-20 02:55:12 -0600

Jakob gravatar image

updated 2020-07-20 07:35:55 -0600

I have a set of points in an image (vector<Point2f>) and I want to calculate their descriptors. I want to use the ORB descriptor for this. It is possible to calculate the descriptors for any set of keypoints (vector<KeyPoint>), but they have additional information like size and rotation (the BRIEF descriptor in ORB is very sensitive to rotation, so it's not appropriate just use default keypoint parameters). The keypoint detector finds this additional information, but I'm curious if there is a way to do what it does when the point coordinate is already known.

The ORB keypoint detector roughly does these steps:

  1. FAST keypoint detection (with image pyramids)
  2. Calculate the Harris corner response (aka corner measure, corner score, corner criterion etc.)
  3. Calculate mass center and inertia in the keypoint's neighborhood to find the rotation

Then the compute-function calculates the rotation-compensated BRIEF descriptor.

In short I wish to circumvent step 1. in the ORB detector. The image pyramid might complicate things, but for my use-case, loosing scale invariance is a reasonable compromise.

Edit: I just found out that the "size" parameter in a keypoint is the size of the patch around the point which is used to compute the descriptor. For ORB this is 31 (i.e. 31x31 pixels). However, this number increases when the "octave" parameter increases. My understanding is that the value of "octave" is in which layer in the image pyramid the keypoint was detected. The patch, I assume, gets scaled and rotated before the descriptor is computed (and I also assume it gets discretized so that there are 31x31 pixels when computing the descriptor).

Anyhow, since size invariance is neglectable in this use case, "size" and "octave" is always 31.0f and 0. The question then is if there is an OpenCV function for calculating the corner response directly, and one for calculating the angle (I have some vague recollection of the required math to do those things, but I assume that an OpenCV implementation will be more optimized).

edit retag flag offensive close merge delete

Comments

Take a look at the source code. For instance, copy and modify the function ICAngles to suit your needs.

Der Luftmensch gravatar imageDer Luftmensch ( 2020-07-20 12:02:58 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2020-07-22 05:40:16 -0600

Jakob gravatar image

Following Der Luftmensh's advice: The functions HarrisResponses and ICAngles in orb.cpp can be copied and modified. Note that these functions do not check whether the points that are used are too close to the image's edge (trying to do a calculation on a patch around e.g. point (0,0) will cause an error).

ICAngles takes an argument vector<int> u_max, which at first can be a little confusing. The function does its calculations on a circular patch inside the square 31x31 patch. u_max tells how many columns away from center in each row of the square patch that constitutes the circle (if you plot the values of u_max in an horizontal bar-diagram, it will resemble a circle quadrant).

Lines 814-829 in orb.cpp calculates u_max, but for a 31x31 patch std::vector<int> u_max = {15, 15, 15, 15, 14, 14, 14, 13, 13, 12, 11, 10, 9, 8, 6, 3, 0}; can be used.

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2020-07-20 02:55:12 -0600

Seen: 1,268 times

Last updated: Jul 22 '20