Find average distance between 2 matched keypoints (after using findHomography() mask) c++

edit

What you're doing is completely fine. You just need to answer to this: which distance you would like to use? There are tons of distances, Euclidean, Mahalanobis, Manatthan distance. You have to choose it. If you choose, for example, a Sum of Squared Distance (SSD), it will be (x-x')^2+(y-y')^2, where (x,y) are the keypoints location in the first image and (x',y') are the matched keypoints in the second image.

If you want to find the average distance along x and y axis, simply defined as you did, then you have to sum all the distance x-x' in a variable and y-y' in another variable and do the average. Like this

int sumXdist = 0;
int sumYdist = 0;
int numInliers=0;

for(size_t i = 0; i < matches.size(); i++)
{
    if(mask.at(i) == 1) //So is an inliers
    {
        numInliers++;
        cv::DMatch first = matches[i][0];
        sumXdist += fabs(keypoints1[first.queryIdx].pt.x - keypoints2[first.trainIdx].pt.x);
        sumYdist += fabs(keypoints1[first.queryIdx].pt.y - keypoints2[first.trainIdx].pt.y);
    }
}
double avgAlongX = (double) sumXdist/numInliers;
double avgAlongY = (double) sumYdist/numInliers;

This code is just an example in C++ but I hope you get it!