Determine the different corners of a rectangle detected on screen

edit

I have an image with a significantly large rectangular area in it, as shown in the image below.

(On an Android phone) I can easily detect the rectangular area and retrieve its 4 corners using Canny edge detector and findContour() in OpenCV. But my goal is:

To have a list of the 4 corners, where the first element is the top-left corner of the rectangle, to later rectify the rectangle correctly.

First, many people suggest that OpenCV findContour() always return a list of corners where the first element is always the one having minimum y pixel values. However, I have checked myself that this is NOT the case for some camera angles.

Second, this problem is trivial if the camera is pointing at the image target in a somewhat top-down view - I can simply find the corner having the smallest (x + y) pixel value. However, in my setting, I allow the camera to point at the target in almost any arbitrary angle, as long as the rectangular contour is detected.

So, the tricky part is, the top-left corner may not have minimum y pixel value, or minimum x pixel value, or whatsoever. I have to also consider the on-screen distances between the corners to determine where the long edges and the short edges are. But then there is the foreshortening problem, i.e. the long edge would appear shorter than the short edge of the rectangle in some camera angles.

Here are my thoughts:

• I should still consider the minimum (x+y) pixel value as one of the approach
• I should consider the on-screen distances between corners, and probably also incorporate the accelerometer values of my phone to have an understanding of my phone's orientation, so as to counter the foreshortening effect.

However, I don't have a solid solution to this yet. Any help and advice would be appreciated.

Edit:

I also have the physical size (width and height) of the rectangular area.

Example:

In the below images, I already got the blue contour and the 4 red corners from findContour(). My goal would be to find out which red dot is the top-left corner of the rectangular area. I know I cannot solely depend on their pixel values.