Combining FAST and SIFT: low Keypoint size [closed]
I try to combine the FAST algorithm and parts of the SIFT Algorithm to get a fast Solution for Mobile Phones. The code I created for that works like the following:
- Get Image
- calculate Size for Resize with
src.rows/factorandsrc.cols/factorwhere factor = sqrt(2) - resize Image (INTER_NEAREST) and Blur it with GaussianKernel(5x5)
- Run FAST over the new smaller and blurred Image
- Iterate through every candidate and make a Image Patch (15x15) with the candidate in its origin
- calculate the OrientationHistogram and get the peaks of it.
Here the Code: Migrated to Github because it is really long
The Idea for this comes from this Paper
The first Steps are running pretty good. But the last one seems to result in ONLY 4 Keypoints while I get 220193 Candidates from FAST.
EDIT I found such a low Keypoint size, because I was inserting the Point Location of the Interest Point in the Source Image not the Image Patch which I pass to the calcOrientationHist Function But now I got another Error look at EDIT2
The Code I am using is similar with the code from calcOrientationHist Function from the sift.cpp The Input Image is my Image Patch, the Point is the Candidates Point. To calculate the radius and sigma I needed scl_octvbut because I dont really go in "Octaves" through the Image I just calculated it like this: float scl_octv = kpt.size * 0.5f / (1 << 1);similar to the sift.cpp but with the difference that I add bytewise 1 and not the actual octave. The Peak detection after creating the histogram looks like this:
Where SIFT_ORI_PEAK_RATIO is 0.8f.
When I cout the size of features I get the number 4 in the console.
After the Image is read through ìmread I turn it into a grayscale image with cvtColor(bas, src, CV_BGR2GRAY);
where bas is the loaded Image and src the output. In my Example I only go through one Resize and Blur step.
EDIT2 After solving this Error, I get a Malloc Error now.
The Position this Error is thrown is at Line 207 in my Code
I already tried to free as much Memory as possible, but I don't believe, that this is the Solution. XCode Analysis says, that my Application needs 9MB memory.
Also I thought, that I occurs, because I call the Pixel value with img.at<sift_wt>(...) where typedef float sift_wt but the Image Patch is from the type CV_8UC1 (console output 0 at patch.type()) But actually I just took that from the calcOrientationHist from the opencv-contrib. Is Sift calculating the Descriptor in the Source Image? It also should be a Grayscale Image or not?
I hope somebody can see the Error Occurs.
EDIT3
After changing the type in the img.at<sift_wt>(...)Position nothing changed. So I googled Solutions and landed at the GuardMalloc feature from XCode. Enabling it showed me a new Error which is probably the Reason I get the Malloc ...
just be aware, that even if you roll your own sift code, the algorithm is still patented (and you might not be allowed to use it, if you intend to sell your software)
Thanks for the Advice @berak but It is not intended to sell the software. Its for a Bachelor Thesis
nvm, then, and good luck with your thesis !
Oh yeah I will need that. Thank you ;)
It's almost always a great experience to write your own algorithm pipelines, but be aware that you are unlikely to beat the performance of the OpenCV canned binary descriptors. SIFT is certainly the most famous, but you might choose instead to try implementing AGAST (pyramid FAST) with rotated (if you desire rotation invariance) BRIEF descriptors (or FREAK, or BRISK). That combination is likely to be the near the fastest combination of current methods (check out ORB), you should then compare performance to a canned SIFT. In general you have two options for registering one image to another, feature driven and dense correspondence. To my knowledge, the feature driven methods do not vary all that much in accuracy, but can vary greatly in computational cost (SIFT being very slow/expensive).
Thanks @Der Luftmensch My 2nd Supervisor has a lot of interest in this Solution here. Also I found my Mistake now why it produced low Keypoint Values in the Feature Vector. I put the Patch into the calcOriHist Function but the Keypoint Location of the original Image. Now I changed it into the centre Point of the Imagepatch (the Keypoint Location is always in the center of the image patch) but now I get an malloc error when I add the candidate Vector in another vector through push_back. Should I open a new Question for that one? Or just edit this Question?
It's usually better to edit/update the question than to ask a new question.
ok, I just edited it now here. I hope one of you sees what is wrong there because I debugged it about 20 times and havent found anything yet.