Check image dissimilar areas using SIFT

asked 2020-10-28 03:28:47 -0500

sachet12345 gravatar image

updated 2020-10-28 03:31:20 -0500

berak gravatar image

I want to see which areas in image are dissimilar. Attaching 2 images in which there is a difference in top left part. It shouldn't show the difference if some part is scaled, translated or skewed. I am trying to use SIFT matching to see the non matched points. image description image description.


import cv2
import numpy as np
from matplotlib import pyplot as plt
from scipy.spatial.distance import euclidean

img1 = cv2.imread('plant.png')
img2 = cv2.imread('plantErase.png')

gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

sift = cv2.SIFT_create()

kp1, des1 = sift.detectAndCompute(gray1, None)
kp2, des2 = sift.detectAndCompute(gray2, None)

im_with_keypoints1 = cv2.drawKeypoints(gray1, kp1, np.array([]))
im_with_keypoints2 = cv2.drawKeypoints(gray2, kp2, np.array([]))


FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)

flann = cv2.FlannBasedMatcher(index_params, search_params)

matches = flann.knnMatch(des1,des2,k=2)

MIN_MATCH_COUNT = 10

good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)


if len(good)>MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)

    kp1_matched=([ kp1[m.queryIdx] for m in good ])
    kp2_matched=([ kp2[m.trainIdx] for m in good ])

    kp1_miss_matched=[kp for kp in kp1 if kp not in kp1_matched]
    kp2_miss_matched=[kp for kp in kp2 if kp not in kp2_matched]

    img1_miss_matched_kp = cv2.drawKeypoints(img1,kp1_miss_matched, None,color=(255,0,0), flags=0)
    plt.imshow(img1_miss_matched_kp),plt.show()

    img2_miss_matched_kp = cv2.drawKeypoints(img2,kp2_miss_matched, None,color=(255,0,0), flags=0)
    plt.imshow(img2_miss_matched_kp),plt.show()
edit retag flag offensive close merge delete

Comments

and what is your actual question here ?

berak gravatar imageberak ( 2020-10-28 03:32:37 -0500 )edit

The non-matched keypoints are not enough to identify which area is different. Ratio of matched and non matched keypoints is almost 1. Ss, I wanted to know how to identify which area is different irrespective of translation/skewing/rotation of some part.

I am new to opencv. I would be thankful if you could provide any resources in this direction.

sachet12345 gravatar imagesachet12345 ( 2020-10-28 03:48:31 -0500 )edit

The non-matched keypoints are not enough to identify which area is different.

this is no surprise, it's not meant to be used like that

irrespective of translation/skewing/rotation of some part.

maybe try to find a homography between the 2 images, using your SIFT matching, then warp the 2nd image to the 1st and use a straight absdiff() to find differences

berak gravatar imageberak ( 2020-10-28 04:55:19 -0500 )edit

Do you mean homography b/w a particular part of both pictures? Could you tell me what module to use to warp? And if absdiff will ignore the parts that are skewed/rotated/translated?

sachet12345 gravatar imagesachet12345 ( 2020-10-28 06:30:43 -0500 )edit

will there be parts in the image that are skewed/rotated/translated? or one whole image moved wrt. the other ? have another look at the tutorials if it'S the latter, you could align both images. using this

berak gravatar imageberak ( 2020-10-28 07:25:00 -0500 )edit

@berak. That tutorial doesn't accurately. The newest OpenCV 4.5.0 version will get u nicely accurately.

supra56 gravatar imagesupra56 ( 2020-10-28 07:38:11 -0500 )edit

what ? try to write a complete emglish sentence, not garbage ...

berak gravatar imageberak ( 2020-10-28 08:30:53 -0500 )edit

Yes, parts of the image will be changed. I understand the latter can be solved by absdiff().

sachet12345 gravatar imagesachet12345 ( 2020-10-28 08:50:12 -0500 )edit

did you try someting like this

import cv2
import numpy as np

img1 = cv2.imread('16038730944851118.png')
img2 = cv2.imread('16038730733833271.png')

img1 = cv2.resize(img1,(img2.shape[1],img2.shape[0]))
diff=cv2.subtract(img1, img2)
cv2.imshow('diff',diff)
cv2.waitKey(0)
sturkmen gravatar imagesturkmen ( 2020-10-28 08:54:16 -0500 )edit

Hi @strukmen, this would show the difference even if say, a rectangle in the whole diagram is moved a little bit to the right when in reality, this is not a difference.

sachet12345 gravatar imagesachet12345 ( 2020-10-28 09:47:42 -0500 )edit