Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Stitch images with OpenCV 3 (+contrib) and Python 2.7

I want to stitch 2 mages using OpenCV 3.0 (with contrib) and Python 2.7.

I have written a program to do this but the result is so bad. I'm using SIFT features to do this. I have displayed the found features and I think they are very good, the problem is the homography. The transformation applied to the images is totally wrong and I don't know why.

Here are the two images that I want to stitch

enter image description here

enter image description here

Here the keypoints

enter image description here

And here the results after applying the homography (and its inverse)

enter image description here enter image description here

What I have tested

Here my program, 90% is copied from this post: http://stackoverflow.com/questions/6542339/calculate-offset-skew-rotation-of-similar-images-in-c/6545111#6545111 I have added two methods explained below.

PATH = "PATH_TO_IMAGES"
SHOW_MATCHES = False

# Load the two images
img1 = cv2.imread(PATH + "image1.jpg", -1)
img2 = cv2.imread(PATH + "image2.jpg", -1)

# Transform to RGB
cv2.cvtColor(img1, cv2.COLOR_BGR2RGB, img1)
cv2.cvtColor(img2, cv2.COLOR_BGR2RGB, img2)

# Get their dimensions
height, width = img1.shape[:2]

# Resize them (they are too big)
img1 = cv2.resize(img1, (width / 4, height / 4))
img2 = cv2.resize(img2, (width / 4, height / 4))

# Get the resized image's dimensions
height, width = img1.shape[:2]


# Initiate SIFT detector
sift = X2D.SIFT_create()

# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)

# BFMatcher with default params
bf = cv2.BFMatcher()

# Note: 'k=2' -> for each keypoint, there can be
# 2 or less matches.
matches = bf.knnMatch(des1,des2, k=2)

###### Here the filter that I have added:
# Filter the matches to remove the wrong ones. This filter 
# removes all pairs of keypoints whose y-coordinate difference
# is lower than the image's height * 0.1.
# The method is implementated below.
matches = filterMatches(kp1, kp2, matches, height, 0.1)


# Apply ratio test
good = []
for m,n in matches:
    if m.distance < 0.75*n.distance:
        good.append([m])

# Get the coordinates of the mattching keypoints:
# This method is implemented below.
sP, dP = Tools.pointsFromMatches(kp1, kp2, matches)

# Get the homography and its inverse:
H, mask = cv2.findHomography(sP, dP, cv2.RANSAC)
iH = np.linalg.inv(H)

# Apply the homography to both images:
alignedImg1 = cv2.warpPerspective(img1, H, (width, height), flags = cv2.INTER_LINEAR + cv2.BORDER_CONSTANT)
alignedImg2ToImg1 = cv2.warpPerspective(img2, iH, (width, height), flags = cv2.INTER_LINEAR + cv2.BORDER_CONSTANT)

# Show the results:
plt.imshow(alignedImg1), plt.show()
plt.imshow(alignedImg2ToImg1), plt.show()

The implementation of my methods

def filterMatches(kp1, kp2, matches, imgHeight, thresFactor = 0.4):
    """
    Removes the matches that correspond to a pair of keypoints (kp1, kp2)
    which y-coordinate difference is lower than imgHeight * thresFactor.

    Args:
        kp1 (array of cv2.KeyPoint): Key Points.

        kp2 (array of cv2.KeyPoint): Key Points.

        matches (array of cv2.DMatch): Matches between kp1 and kp2.

        imgHeight (Integer): height of the image that has produced kp1 or kp2.

        thresFactor (Float): Use to calculate the threshold. Threshold is 
            imgHeight * thresFactor.

    Returns:
        array of cv2.DMATCH: filtered matches.

    """
    filteredMatches = [None]*len(matches)
    counter = 0
    threshold = imgHeight * thresFactor
    for i in range(len(kp1)):
        srcPoint = kp1[ matches[i][0].queryIdx ].pt
        dstPoint = kp2[ matches[i][0].trainIdx ].pt
        diff = abs(srcPoint[1] - dstPoint[1])
        if( diff < threshold):
            filteredMatches[counter] = matches[i]
            counter += 1

    return filteredMatches[:counter]


def pointsFromMatches(kp1, kp2, matches):
    """
    Retrieves the pairs (a, b) that meet the condition:

        Given i,j such that kp1[i] matches with kp2[j]:

        a = coordinates of kp1[i].
        b = coordinates of kp2[j].      


    This pairs are returned as the following vectors:

        sP = [a0, a1, a2, ..., an]
        dP = [b0, b1, b2, ..., bn]


    Args:
        kp1 (vector of cv2.KeyPoint): KeyPoints of an image.
        kp2 (vector of cv2.KeyPoint): KeyPoints of an image.
        matches (vector of cv2.DMatch): Matches between kp1 and kp2.


    Returs:
        2 vectors (ndarray, ndarray).

    """
    pairsOfKp1 = [i[0].queryIdx for i in matches]
    pairsOfKp2 = [i[0].trainIdx for i in matches]
    sP = cv2.KeyPoint_convert(kp1, pairsOfKp1)
    dP = cv2.KeyPoint_convert(kp2, pairsOfKp2)
    return sP, dP