I have to stitch two images using OpenCV. I have very good KeyPoints detected, what I need to do is to find and apply the transformation. I have tried it but I get bad results.

What I have tested

At the begining, I tested with findHomography() and warpPerspective(). The result of this is bad because this method applies perspective transformation, and I need to avoid it (I only want to translate, rotate and stretch the images)

Second, I have tested with getAffineTransform() and warpAffine(). The affine transformation applied to the first image is good for me but I need to transform also the second image to create the panorama. This is what I don't know how to do.

I think that I need get the inverse of the transformation matrix (A) and apply it to the second image, but, since the transformation matrix is 2x3 (rows x columns), I calculate the pseudo-inverse (iA). Then I calculate A2 = A * iA and try to apply this to the second image.

If I try to use this matrix, I get an error because the previous result is a 2x2 matrix, that I can't use with warpAffine(), so I don't know how to apply the correct transformation to the second image.

Here is the code, I only want to show both transformed images:

def testAffine(img1, img2, kp1, kp2, good):
    """
    Args:
        img1: Image 1
        img2: Image 2
        kp1: KeyPoints of image 1.
        kp2: KeyPoints of image 2.
        good: matches between kp1 and kp2.

    """
    height, width = img1.shape[:2]

    # Get matching keypoints coordinates
    pairsOfKp1 = [i[0].queryIdx for i in matches]
    pairsOfKp2 = [i[0].trainIdx for i in matches]
    sP = cv2.KeyPoint_convert(kp1, pairsOfKp1)
    dP = cv2.KeyPoint_convert(kp2, pairsOfKp2)

    A = cv2.getAffineTransform(sP, dP)
    iA = np.linalg.pinv(A)
    A2 = np.dot(A, iA)

    R1 = cv2.warpAffine(img1, A, (width, height))
    R2 = cv2.warpAffine(img2, iA, (width, height))   # ERROR: iA is 2x2

    Tools.displayImage(R1)   # this shows the image
    Tools.displayImage(R2)   # this shows the image

Obtained transformation matrixes

A = 
[[  8.97358731e-01   2.55456505e-02   2.44530594e+02]
 [  4.78631544e-03   9.91320045e-01   1.06040609e+01]]

iA = 
[[  1.52255408e-03  -3.47642421e-02]
 [ -4.37411318e-02   1.00868634e+00]
 [  4.08845004e-03   2.21994614e-05]]

A * iA = 
[[  1.00000000e+00  -2.45376636e-15]
 [ -1.38777878e-17   1.00000000e+00]]

Since the A * iA matrix is 2x2 and I need a 2x3 matrix to apply the affine trsansform

I have found the solution. First of all, according to drawMatchesKnn documentation:

keypoints1[i] has a corresponding point in keypoints2[matches[i]]

In my code, 'keypoints1' is kp1, 'keypoints2' is kp2 and 'matches' is matches.

The correspondency between kp1 and kp2 is: kp1[i] matches with kp2[ matches[i].trailIdx ].

Here the finally function which filter the keypoints removing all of them which y-coordinate is higher than the image's height * n, where n is a given number (between 0 and 1):

def filterMatches(kp1, kp2, matches, imgHeight, thresFactor = 0.4):
"""
Removes the matches that correspond to a pair of keypoints (kp1, kp2)
which y-coordinate difference is lower than imgHeight * thresFactor.

Args:
    kp1 (array of cv2.KeyPoint): Key Points.

    kp2 (array of cv2.KeyPoint): Key Points.

    matches (array of cv2.DMATCH): Matches between kp1 and kp2.

    imgHeight (Integer): height of the image that has produced kp1 or kp2.

    thresFactor (Float): Use to calculate the threshold. Threshold is 
        imgHeight * thresFactor.

Returns:
    array of cv2.DMATCH: filtered matches.

"""
filteredMatches = [None]*len(matches)
counter = 0
threshold = imgHeight * thresFactor
for i in range(len(kp1)):
    srcPoint = kp1[ matches[i][0].queryIdx ].pt
    dstPoint = kp2[ matches[i][0].trainIdx ].pt
    diff = abs(srcPoint[1] - dstPoint[1])
    if( diff < threshold):
        filteredMatches[counter] = matches[i]
        counter += 1

return filteredMatches[:counter]

I have the SIFT keypoints of two images (calculated with Python + OpenCV 3). I want to filter them by their y-coordinate.

Specifically, I want to remove all matching points whose difference of y-coordinate is higher than the image height divided by 10, for example: If two matching points are A(x1, y1) and B(x2, y2):

if abs(y2 - y1) > imageHeight / 10 then remove that maching points.

What I have test

Here is the code I have tested. I'm removing keypoints, but not what I want to remove.

# Load the two images
img1 = cv2.imread(PATH + "image1.jpg", -1)
img2 = cv2.imread(PATH + "image2.jpg", -1)


# Get their dimensions
height, width = img1.shape[:2]

# Resize them (they are too big)
img1 = cv2.resize(img1, (width / 4, height / 4))
img2 = cv2.resize(img2, (width / 4, height / 4))

# Get the resized image's dimensions
height, width = img1.shape[:2]

# Initiate SIFT detector
sift = X2D.SIFT_create()

# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)

# BFMatcher with default params
bf = cv2.BFMatcher()
matches = bf.knnMatch(des1,des2, k=2)

### Here the filtering attempt ###
# Alloc a new vector for filtered matches
filteredMatches = [None] * len(matches)
# Counter that will count how many matches I have at the end
counter = 0

# for each match
for i in range(len(matches)):

    # Get the "img1" heypoint
    leftPoint = kp1[ matches[i][0].queryIdx ].pt   #'left' image
    # Get the "img2" keypoint
    rightPoint = kp2[ matches[i][0].trainIdx ].pt  #'right' image

    # substract the y-coordinate of both points and compare
    # with height / 10
    if( abs(leftPoint[1] - rightPoint[1]) < height / 10):
        # if the difference is lower than higher / 10, add it
        # to the new list and increment the counter:
        filteredMatches[counter] = matches[i]
        counter += 1

# fix the filtered list size
matches = matches[:counter]

I'm not sure if I'm using queryIdx and trainIdxcorrectly, but according with this post (http://stackoverflow.com/questions/10...) I think so.