I want to stitch 2 mages using OpenCV 3.0 (with contrib) and Python 2.7.
I have written a program to do this but the result is so bad. I'm using SIFT features to do this. I have displayed the found features and I think they are very good, the problem is the homography. The transformation applied to the images is totally wrong and I don't know why.
Here are the two images that I want to stitch
Here the keypoints
And here the results after applying the homography (and its inverse)
What I have tested
Here my program, 90% is copied from this post: http://stackoverflow.com/questions/6542339/calculate-offset-skew-rotation-of-similar-images-in-c/6545111#6545111 I have added two methods explained below.
PATH = "PATH_TO_IMAGES"
SHOW_MATCHES = False
# Load the two images
img1 = cv2.imread(PATH + "image1.jpg", -1)
img2 = cv2.imread(PATH + "image2.jpg", -1)
# Transform to RGB
cv2.cvtColor(img1, cv2.COLOR_BGR2RGB, img1)
cv2.cvtColor(img2, cv2.COLOR_BGR2RGB, img2)
# Get their dimensions
height, width = img1.shape[:2]
# Resize them (they are too big)
img1 = cv2.resize(img1, (width / 4, height / 4))
img2 = cv2.resize(img2, (width / 4, height / 4))
# Get the resized image's dimensions
height, width = img1.shape[:2]
# Initiate SIFT detector
sift = X2D.SIFT_create()
# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)
# BFMatcher with default params
bf = cv2.BFMatcher()
# Note: 'k=2' -> for each keypoint, there can be
# 2 or less matches.
matches = bf.knnMatch(des1,des2, k=2)
###### Here the filter that I have added:
# Filter the matches to remove the wrong ones. This filter
# removes all pairs of keypoints whose y-coordinate difference
# is lower than the image's height * 0.1.
# The method is implementated below.
matches = filterMatches(kp1, kp2, matches, height, 0.1)
# Apply ratio test
good = []
for m,n in matches:
if m.distance < 0.75*n.distance:
good.append([m])
# Get the coordinates of the mattching keypoints:
# This method is implemented below.
sP, dP = Tools.pointsFromMatches(kp1, kp2, matches)
# Get the homography and its inverse:
H, mask = cv2.findHomography(sP, dP, cv2.RANSAC)
iH = np.linalg.inv(H)
# Apply the homography to both images:
alignedImg1 = cv2.warpPerspective(img1, H, (width, height), flags = cv2.INTER_LINEAR + cv2.BORDER_CONSTANT)
alignedImg2ToImg1 = cv2.warpPerspective(img2, iH, (width, height), flags = cv2.INTER_LINEAR + cv2.BORDER_CONSTANT)
# Show the results:
plt.imshow(alignedImg1), plt.show()
plt.imshow(alignedImg2ToImg1), plt.show()
The implementation of my methods
def filterMatches(kp1, kp2, matches, imgHeight, thresFactor = 0.4):
"""
Removes the matches that correspond to a pair of keypoints (kp1, kp2)
which y-coordinate difference is lower than imgHeight * thresFactor.
Args:
kp1 (array of cv2.KeyPoint): Key Points.
kp2 (array of cv2.KeyPoint): Key Points.
matches (array of cv2.DMatch): Matches between kp1 and kp2.
imgHeight (Integer): height of the image that has produced kp1 or kp2.
thresFactor (Float): Use to calculate the threshold. Threshold is
imgHeight * thresFactor.
Returns:
array of cv2.DMATCH: filtered matches.
"""
filteredMatches = [None]*len(matches)
counter = 0
threshold = imgHeight * thresFactor
for i in range(len(kp1)):
srcPoint = kp1[ matches[i][0].queryIdx ].pt
dstPoint = kp2[ matches[i][0].trainIdx ].pt
diff = abs(srcPoint[1] - dstPoint[1])
if( diff < threshold):
filteredMatches[counter] = matches[i]
counter += 1
return filteredMatches[:counter]
def pointsFromMatches(kp1, kp2, matches):
"""
Retrieves the pairs (a, b) that meet the condition:
Given i,j such that kp1[i] matches with kp2[j]:
a = coordinates of kp1[i].
b = coordinates of kp2[j].
This pairs are returned as the following vectors:
sP = [a0, a1, a2, ..., an]
dP = [b0, b1, b2, ..., bn]
Args:
kp1 (vector of cv2.KeyPoint): KeyPoints of an image.
kp2 (vector of cv2.KeyPoint): KeyPoints of an image.
matches (vector of cv2.DMatch): Matches between kp1 and kp2.
Returs:
2 vectors (ndarray, ndarray).
"""
pairsOfKp1 = [i[0].queryIdx for i in matches]
pairsOfKp2 = [i[0].trainIdx for i in matches]
sP = cv2.KeyPoint_convert(kp1, pairsOfKp1)
dP = cv2.KeyPoint_convert(kp2, pairsOfKp2)
return sP, dP