Help with OpenCV image classification by ORB features

asked 2020-04-27 03:16:50 -0600

updated 2020-05-03 04:22:23 -0600

Hi all. I need help with classifying images by extracted ORB features (vectors). I have 3 etalon images. I have to classify images by ORB functions (compare other images with these etalons). In the first, I tried the next solution:

 #get etalons (files)
   camera = cv2.imread('/content/drive/My Drive/ColabNotebooks/images/etalons/camera.jpg')
   phone = cv2.imread('/content/drive/My Drive/ColabNotebooks/images/etalons/phone.jpg')
   lamp = cv2.imread('/content/drive/My Drive/ColabNotebooks/images/etalons/lamp.jpg')

   gray_camera = cv2.cvtColor(camera, cv2.COLOR_BGR2GRAY)
   gray_phone = cv2.cvtColor(phone, cv2.COLOR_BGR2GRAY)
   gray_lamp = cv2.cvtColor(lamp, cv2.COLOR_BGR2GRAY)

   # find the keypoints with ORB and compute the descriptors with ORB
   camera_orb_kp, camera_des = orb.detectAndCompute(gray_camera, None)
   phone_orb_kp, phone_des = orb.detectAndCompute(gray_phone, None)
   lamp_orb_kp, lamp_des = orb.detectAndCompute(gray_lamp, None)

   # draw only keypoints location,not size and orientation
   camera2 = cv2.drawKeypoints(gray_camera,camera_orb_kp,outImage=None, color=(0,255,0), flags=0)
   phone2 = cv2.drawKeypoints(gray_phone,phone_orb_kp,outImage=None, color=(0,255,0), flags=0)
   lamp2 = cv2.drawKeypoints(gray_lamp,lamp_orb_kp,outImage=None, color=(0,255,0), flags=0)
   plt.title('Camera Etalon ORB Features'),plt.imshow(camera2),plt.show()
   plt.title('Phone Etalon ORB Features'),plt.imshow(phone2),plt.show()
   plt.title('Lamp Etalon ORB Features'),plt.imshow(lamp2),plt.show()

control points ORB

img_test = cv2.imread('/content/drive/My Drive/ColabNotebooks/images/image_0045.jpg')
gray_test = cv2.cvtColor(img_test, cv2.COLOR_BGR2GRAY)

# find the keypoints with ORB and compute the descriptors with ORB
kpt, dest = orb.detectAndCompute(gray_test, None)

# ORB
img2_test = cv2.drawKeypoints(gray_test,kpt,outImage=None, color=(0,255,0), flags=0)
plt.title('First test image ORB features'),plt.imshow(img2_test),plt.show()

First sample image ORB control points

# ORB matches
camera_matches = bf.match(camera_des,dest)
phone_matches = bf.match(phone_des,dest)
lamp_matches = bf.match(lamp_des,dest)

# Sort them in the order of their distance.
camera_matches = sorted(camera_matches, key = lambda x:x.distance)
phone_matches = sorted(phone_matches, key = lambda x:x.distance)
lamp_matches = sorted(lamp_matches, key = lambda x:x.distance)
# print('matches/kp:')
print('ORB matches: ')
print(len(camera_matches), len(phone_matches), len(lamp_matches))

Results

But the classification was incorrect (image with camera was classified as lamp). I not understand which way to compare ORB functions is correct. Please help me with this task. Describe me how I should compare control poins and descriptors. Thanks in advance.

edit retag flag offensive close merge delete

Comments

classifying images by ORB functions.

trying to abuse opencv's feature matching functions will not work, they're not made for this purpose.

I tried the following solution.

please put your code here, your colab won't be there in the future

berak gravatar imageberak ( 2020-04-28 06:21:59 -0600 )edit

again, the purpose of feature matching is to get a homography between 2 parts of the same image (or scene), NOT to retrieve some kind of image similarity.

your matches are meaningless for classification, and the length of the matches is arbitrary.

I have to classify images by ORB

you probably don't have to, and you clearly shouldn't.

berak gravatar imageberak ( 2020-05-03 04:57:27 -0600 )edit