Ask Your Question

Combining FAST and SIFT: low Keypoint size [closed]

asked 2016-09-27 02:43:06 -0500

Vintez gravatar image

updated 2016-09-29 06:43:21 -0500

I try to combine the FAST algorithm and parts of the SIFT Algorithm to get a fast Solution for Mobile Phones. The code I created for that works like the following:

  • Get Image
  • calculate Size for Resize with src.rows/factor and src.cols/factorwhere factor = sqrt(2)
  • resize Image (INTER_NEAREST) and Blur it with GaussianKernel(5x5)
  • Run FAST over the new smaller and blurred Image
  • Iterate through every candidate and make a Image Patch (15x15) with the candidate in its origin
  • calculate the OrientationHistogram and get the peaks of it.

Here the Code: Migrated to Github because it is really long

The Idea for this comes from this Paper

The first Steps are running pretty good. But the last one seems to result in ONLY 4 Keypoints while I get 220193 Candidates from FAST.

EDIT I found such a low Keypoint size, because I was inserting the Point Location of the Interest Point in the Source Image not the Image Patch which I pass to the calcOrientationHist Function But now I got another Error look at EDIT2

The Code I am using is similar with the code from calcOrientationHist Function from the sift.cpp The Input Image is my Image Patch, the Point is the Candidates Point. To calculate the radius and sigma I needed scl_octvbut because I dont really go in "Octaves" through the Image I just calculated it like this: float scl_octv = kpt.size * 0.5f / (1 << 1);similar to the sift.cpp but with the difference that I add bytewise 1 and not the actual octave. The Peak detection after creating the histogram looks like this:

Where SIFT_ORI_PEAK_RATIO is 0.8f. When I cout the size of features I get the number 4 in the console.

After the Image is read through ìmread I turn it into a grayscale image with cvtColor(bas, src, CV_BGR2GRAY);
where bas is the loaded Image and src the output. In my Example I only go through one Resize and Blur step.

EDIT2 After solving this Error, I get a Malloc Error now.

The Position this Error is thrown is at Line 207 in my Code

I already tried to free as much Memory as possible, but I don't believe, that this is the Solution. XCode Analysis says, that my Application needs 9MB memory.

Also I thought, that I occurs, because I call the Pixel value with<sift_wt>(...) where typedef float sift_wt but the Image Patch is from the type CV_8UC1 (console output 0 at patch.type()) But actually I just took that from the calcOrientationHist from the opencv-contrib. Is Sift calculating the Descriptor in the Source Image? It also should be a Grayscale Image or not?

I hope somebody can see the Error Occurs.

EDIT3 After changing the type in the<sift_wt>(...)Position nothing changed. So I googled Solutions and landed at the GuardMalloc feature from XCode. Enabling it showed me a new Error which is probably the Reason I get the Malloc ... (more)

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by Vintez
close date 2016-12-01 12:17:46.385166


just be aware, that even if you roll your own sift code, the algorithm is still patented (and you might not be allowed to use it, if you intend to sell your software)

berak gravatar imageberak ( 2016-09-28 02:59:36 -0500 )edit

Thanks for the Advice @berak but It is not intended to sell the software. Its for a Bachelor Thesis

Vintez gravatar imageVintez ( 2016-09-28 03:08:43 -0500 )edit

nvm, then, and good luck with your thesis !

berak gravatar imageberak ( 2016-09-28 03:16:25 -0500 )edit

Oh yeah I will need that. Thank you ;)

Vintez gravatar imageVintez ( 2016-09-28 03:19:17 -0500 )edit

It's almost always a great experience to write your own algorithm pipelines, but be aware that you are unlikely to beat the performance of the OpenCV canned binary descriptors. SIFT is certainly the most famous, but you might choose instead to try implementing AGAST (pyramid FAST) with rotated (if you desire rotation invariance) BRIEF descriptors (or FREAK, or BRISK). That combination is likely to be the near the fastest combination of current methods (check out ORB), you should then compare performance to a canned SIFT. In general you have two options for registering one image to another, feature driven and dense correspondence. To my knowledge, the feature driven methods do not vary all that much in accuracy, but can vary greatly in computational cost (SIFT being very slow/expensive).

Der Luftmensch gravatar imageDer Luftmensch ( 2016-09-28 07:01:39 -0500 )edit

Thanks @Der Luftmensch My 2nd Supervisor has a lot of interest in this Solution here. Also I found my Mistake now why it produced low Keypoint Values in the Feature Vector. I put the Patch into the calcOriHist Function but the Keypoint Location of the original Image. Now I changed it into the centre Point of the Imagepatch (the Keypoint Location is always in the center of the image patch) but now I get an malloc error when I add the candidate Vector in another vector through push_back. Should I open a new Question for that one? Or just edit this Question?

Vintez gravatar imageVintez ( 2016-09-28 09:10:54 -0500 )edit

It's usually better to edit/update the question than to ask a new question.

Der Luftmensch gravatar imageDer Luftmensch ( 2016-09-28 12:50:50 -0500 )edit

ok, I just edited it now here. I hope one of you sees what is wrong there because I debugged it about 20 times and havent found anything yet.

Vintez gravatar imageVintez ( 2016-09-29 03:39:06 -0500 )edit

1 answer

Sort by » oldest newest most voted

answered 2016-09-29 07:36:01 -0500

Vintez gravatar image

So I found the Solution for both Errors now.

Low KeyPoint Value The fault I made was, to pass the Point Value of the KeyPoint to the calcOrientationHist(...) function, while I pass the Image Patch and not the Source Image of the KeyPoint. As the Paper says, the Origin/centre of each Image Patch should be the KeyPoint, so I passed the centre Point of the Image patch as Point to the function. After that I got a good Value of KeyPoints per scale Step.

Malloc Error To solve a Malloc Error I recommend for XCode Users to always activate the GuardMalloc feature for the Debugger. You can Access this Feature through the Run Options of your Application under the tab "Diagnostics". The Guard Malloc helps to find the exact Location where the Malloc Error is generated. Without it, it the Malloc Error will occur while running at Line which does not produce the mistake (like in my case line 207 was shown to throw the Error instead it was a completely other line). With a little bit more Debugging it showed, that The Error occured, because I tried to access a value of an array but the index was out of the Arrays bounds. (Temphist.size would be 38, it tried to access the value at 52). The Solution was, that the calculation of Bin was wrong instead of this: cvRound((n/360.f)+ori[k]it should be cvRound((n/360.f)*ori[k]so instead of the value 52 the result would be 8.8

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2016-09-27 02:43:06 -0500

Seen: 251 times

Last updated: Sep 29 '16