Ask Your Question
0

Finding corner points for perspective transform without clear landmarks.

asked 2020-07-29 04:56:33 -0500

o.k gravatar image

updated 2020-07-29 10:16:52 -0500

Hi, I posted this question two days ago, which leaded me to a new problem to solve.

I need to define 4 trapezoid corner points on an image (golf lawn) without any distinct landmarks on it. Except the balls and the flag. All visible balls and the flag must be in the trapezoid.

@kbarni pointed me to some possible solutions but I'm not sure what to do now.

Also I'm pretty bad at math and I have a hard time understanding the formulas explaining computer vision concepts.

Here are my thoughts about how I could eventually get to the corner points but I have no idea how to do it or if it's even possible.


Input

This is the image of interest:

image description


Goal

The goal is to find the corners with the image as is (no extra markers placed in real life).

These are approximately the corners I'm looking for, here I've set the corners by hand which of course isn't 100% accurate.

The corners will be used with getPerspectiveTransform and warpPerspective to get a top down view.

I don't necessarily need the extra padding between the outer most balls and the lines.

image description


Solution?

I do know some variables from the image. I do some balls detection using YOLO and mark them. The model does a decent job at the moment and will get better so that the balls will be market pretty accurately (I hope).

image description

With the detection I can get/know:

  • The balls width/height in pixels/mm
  • The balls centroids

At the moment I don't detect the flag but I will soon.

In real life a golf ball has a diameter of 42.67mm and the hole 107.95mm


What I was thinking of is that I could get a trapezoid from the nearest (biggest) ball bottom corners to the furthest (smallest) ball bottom corners (or something similar) and somehow transform and apply it to the group and align it correctly to get my goal corner points?

Is this in any way possible with the given variables and what do you think about this idea?

Please tell me if you guys need more information.

Help would be greatly appreciated!

Edit/Results

Here are my results after applying @kbarni s answer below and they are pretty awesome imo, even if there is still some work pending to get this done perfectly.

I changed the provided formula a little bit:

(0, C.y),(W,C.y),(W/2+R*W,F.y),(W/2-R*W,F.y)

Where C.y is taken from a bottom corner and F.y from a top corner to have all balls inside the trapezoid.

image description

And here is the top down view result at the moment.

image description

edit retag flag offensive close merge delete

Comments

I think Yolo is a bit overkill for ball detection. You might try HoughCircles, or the inRange function and then ConnectedComponentsWithStats.

The idea you ha

kbarni gravatar imagekbarni ( 2020-07-29 06:00:38 -0500 )edit

Hi @kbarni, thx for responding (again). Yes I wasn't sure if YOLO is the right approach. I tried HoughCircles with Canny and GaussianBlur before but I had difficulties to find (automatically) the right parameters for different images (Different greens etc.). This is why I switched to YOLO to get good results with a lot of different conditions (light, weather etc.). Is there a part of your comment missing about my idea?

o.k gravatar imageo.k ( 2020-07-29 06:12:31 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2020-07-29 06:19:27 -0500

kbarni gravatar image

I think Yolo is a bit overkill for ball detection, but you can keep if it works. You might also try HoughCircles.

The idea you had is clever, but unfortunately there's a risk of having important errors. The problem is that a 1 pixel error in the diameter detection can lead to large errors in distance estimation (~0.5-1m). (and if you zoom in to a far ball, you can see its pixel level contour is not nice)

Anyway, what you can try is to take a ball that's close (C) and another one that's far (F). The corners of the trapezoid are: (0, C.y),(W,C.y),(W/2+R*W2,F.y),(W/2-R*W2,F.y) where R=F.d/C.d.

edit flag offensive delete link more

Comments

Ok I'll do some trials and give feedback. Thx again

o.k gravatar imageo.k ( 2020-07-29 06:47:55 -0500 )edit

This is awesome thank you! I think this goes in the right direction but my results aren't perfect. What do you think about the results (see my edit above) and what should I look out for in order to make it better (except the diameter calculation)?

o.k gravatar imageo.k ( 2020-07-29 10:20:39 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-07-29 04:56:33 -0500

Seen: 71 times

Last updated: Jul 29