Ask Your Question
-1

Determine the different corners of a rectangle detected on screen

asked 2018-01-05 03:32:57 -0500

stephen-mhc gravatar image

updated 2018-01-05 22:32:53 -0500

I have an image with a significantly large rectangular area in it, as shown in the image below.

C:\fakepath\paper_new.png

(On an Android phone) I can easily detect the rectangular area and retrieve its 4 corners using Canny edge detector and findContour() in OpenCV. But my goal is:

To have a list of the 4 corners, where the first element is the top-left corner of the rectangle, to later rectify the rectangle correctly.

First, many people suggest that OpenCV findContour() always return a list of corners where the first element is always the one having minimum y pixel values. However, I have checked myself that this is NOT the case for some camera angles.

Second, this problem is trivial if the camera is pointing at the image target in a somewhat top-down view - I can simply find the corner having the smallest (x + y) pixel value. However, in my setting, I allow the camera to point at the target in almost any arbitrary angle, as long as the rectangular contour is detected.

So, the tricky part is, the top-left corner may not have minimum y pixel value, or minimum x pixel value, or whatsoever. I have to also consider the on-screen distances between the corners to determine where the long edges and the short edges are. But then there is the foreshortening problem, i.e. the long edge would appear shorter than the short edge of the rectangle in some camera angles.

Here are my thoughts:

  • I should still consider the minimum (x+y) pixel value as one of the approach
  • I should consider the on-screen distances between corners, and probably also incorporate the accelerometer values of my phone to have an understanding of my phone's orientation, so as to counter the foreshortening effect.

However, I don't have a solid solution to this yet. Any help and advice would be appreciated.

Edit:

I also have the physical size (width and height) of the rectangular area.

Example:

In the below images, I already got the blue contour and the 4 red corners from findContour(). My goal would be to find out which red dot is the top-left corner of the rectangular area. I know I cannot solely depend on their pixel values.

example1

example2

edit retag flag offensive close merge delete

Comments

minAreaRect(contour) will give you the 4 corners easily, even it there is some rotation

berak gravatar imageberak ( 2018-01-05 11:33:49 -0500 )edit

But I believe minAreaRect(contour) would return the corners of the bounding rectangle, but not my rectangle in the image. In my situation, the camera may be pointing at the image in different angles in 3D, so my rectangle would be somehow distorted. My goal would be to locate the top-left corner in the screen pixel space. But thanks for your response.

stephen-mhc gravatar imagestephen-mhc ( 2018-01-05 22:20:17 -0500 )edit

Have you made any progress on this? I am facing the same issue.

momentuuum gravatar imagemomentuuum ( 2019-10-10 15:27:39 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2018-01-06 00:12:59 -0500

edit flag offensive delete link more

Comments

Thanks for the response. I have checked that and I saw most of the solutions depend on the pixel values of the 4 corners, which generally works. However, as I have stated in my question, my camera may point at the paper at different angles, so sometimes the paper may appear rotated, so that the 2 corners having minimum y pixel values are actually from the side, not from the top. I will continue to work on this, and thank you for your help.

stephen-mhc gravatar imagestephen-mhc ( 2018-01-12 02:47:10 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2018-01-05 03:32:57 -0500

Seen: 1,169 times

Last updated: Jan 05 '18