Combining KAZE keypoints with SIFT descriptors?

asked 2019-11-27 12:41:43 -0500

Karido gravatar image

updated 2019-11-27 12:58:22 -0500

Hello everyone,

is it valid to do the following? (opencv and python)

I detect keypoints using KAZE:

import cv2 as cv    
kaze = cv.KAZE_create()
kp = kaze.detect(img, None)

Now I use my Kaze keypoints and create SIFT descriptors for the given keypoints in the following way:

sift = cv.xfeatures2d.SIFT_create()
kp, des = sift.compute(img, kp)

Finally, I get the keypoints that I first detected with KAZE and have my additional SIFT descriptors. So does the "kp, des = sift.compute(img, kp)" compute an orientation for the given keypoint in a SIFT-manner using it for the descriptor to obtain rotation invariance? Or is it using the orientation of the KAZE-keypoint data type?

Additionally, is the same way of computation valid for the combination of MSER keypoints and SIFT descriptors?

edit retag flag offensive close merge delete

Comments

1

So does the "kp, des = sift.compute(img, kp)" compute an orientation ?

NO. that's handled in the keypoint extraction phase (no matter if it's kaze or sift)

(and to my knowledge, kaze keypoints only work correctly with (A)KAZE descriptors.)

is it valid

the code is, the idea behind it might not hold

berak gravatar imageberak ( 2019-11-27 12:52:28 -0500 )edit

Thanks for your comment. So, it seems that the SIFT descriptors computed for KAZE keypoints lose their rotation invariance due to the absence of the orientation angle of the keypoint?! Because the keypoint of SIFT comes with an orientation angle "kp.angle" not equal to 0 and for KAZE keypoints this angle is just always 0.

Maybe it would be better to raise a warning and add something to the docs if this combination looses rotational invariance?

Karido gravatar imageKarido ( 2019-11-27 13:16:58 -0500 )edit

Interestingly, KAZE + SIFT yield more robust matches after RANSAC than SIFT + SIFT for my used images.

Karido gravatar imageKarido ( 2019-11-27 13:20:08 -0500 )edit