### Why homgraphy matrix doesn't work for any point in the image?

I'm trying to find homography between the following two images.

If I calculate homography based on only corresponding points on the ground then the transformation works well only for the points on the ground.
If I calculate homography based on only corresponding points on the heads then the transformation works well only for the points on the head level.

How do I get homography that will work for any point on the image? Is it even possible ?

**The code for homography calculation:**

```
vector<pair<Point, Point> > homographyPoints;
vector<cv::Point2f> plane1Points;
vector<cv::Point2f> plane2Points;
for (auto& p : homographyPoints)
{
plane1Points.push_back(p.first);
plane2Points.push_back(p.second);
}
homography = findHomography(plane1Points, plane2Points, CV_RANSAC);
```

**The code for using the homography:**

```
vector<cv::Point2f> srcPoints, dstPoints;
srcPoints.push_back(selectedPoint);
cv::perspectiveTransform(srcPoints, dstPoints, params->homography);
```