Ask Your Question
1

How would you stitch those two images as if they are taken from single camera just from the mid side of the field?

asked 2014-07-20 06:08:07 -0500

erogol gravatar image

image description image description

I tried many of the ordinary stitching procedures based on feature matching between the overlapping regions of both images and estimating Homography through those matches. However I could not get very accurate results even I tried different estimation algorithms like RANSAC, Norm 8 Points or others.

Another problem here, I should get a stitched image just like it is taken from a single camera at the mid side of the field. Therefore if I project for example first image to the other via estimated Homography it scewed the first image so the final image as well. I guess for good quality I need to project both images relative to other so that they match each other at some point.

Any suggestion or comment is very welcome...

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2014-07-20 07:37:46 -0500

rwong gravatar image

(This is not an answer; it is meant to point out the difficulties in this task.)


Disclaimer.

I have not actually used the OpenCV stitching library myself; however I studied the algorithms while in school with a MATLAB implementation written by someone else.


My view of the situation.

This seems to be a case that is difficult to process automatically. A lot of manual work is needed. If you have access to MATLAB it will be easier. If you don't, it is quite difficult, or maybe you'll need to do a lot of math yourself (i.e. pen and paper).


Problem 1 - too little overlap between the two views.

Stitching is usually done when the views contain significant overlap. For example, if there is a sequence of views of a panorama, then it is expected that view 1 and view 2 will overlap significantly; so are view 2 and view 3; and so on. Taken together these views span a large area of the scenery.


Problem 2 - using geometrical symmetry instead of point correspondence.

One could argue that they can still be stitched together, based on a human's intuition:

  • The midfield line (and geometrically, the two endpoints that mark the line segment) is visible in both images.
  • Two of the four corners of the field appear in each image.
  • Straight lines can be drawn so that the corners and the endpoints of the midfield line form straight lines.
  • These straight lines are either parallel or perpendicular to each other.

Given the above geometrical symmetry, it is sufficient for a 3D geometry or computer vision student to calculate the homography. (A computer algebra or matrix-solving system such as MATLAB may be needed to solve 4x4 matrices, but that's it.)

Unfortunately, this is not the usage scenario that is targeted by the stitching library. The stitching library is focused primarily on point-based correspondence. It is intended to support both automatically-established point correspondence (RANSAC, etc), or manually added point pairs (with a GUI).

The "outside the box" knowledge that a human has of a soccer field cannot be easily described to the point-based correspondence library.


Solving the problem with human insight.

Realize that one can find a transformation matrix between

  • A quadrilateral ( (x11, y11), (x12, y12), (x21, y21), (x22, y22) ) lying on a 2D plane, and
  • Another quadrilateral ( (u11, v11), (u12, v12), (u21, v21), (u22, v22) ) lying on another 2D plane.

Projection is involved, so the transformation matrix looks like http://research.microsoft.com/pubs/75695/Szeliski-FnT06.pdf

Also realize that you can set the second quadrilateral anyway you want. For example, if you know the widths and lengths of a real soccer field (e.g. in meters or feet), you can scale these numbers and solve a transformation matrix between the image coordinates and the soccer field coordinates.

Stitching the two images is, thus,

  • Transform the soccer field as seen on the first image so that the left side of the soccer field becomes a rectangle. The rectangle ...
(more)
edit flag offensive delete link more

Comments

I am also working through MATLAB and change the code OpenCV when it is ready. Sure using predefiend shape information is very useful idea but again the problem is, for each stadium we need to measure the distance to Camera to estimate the necessary shape of the final field view. Moreover, field scales are not constant and changing place to place. Therefore, predefining expected field shape then introduces additional hand made problems. But very helpful thread thanks... I also read the document you linked. :)

erogol gravatar imageerogol ( 2014-07-21 02:29:12 -0500 )edit

Problem 1 - too little overlap: I don't think the overlap is too little. The images seem of quite large resolution. However, you should restrict matching to a ROI of expected overlap. I think it's around 25% of the image.

Problem 2 - using geometrical symmetry: I have the feeling there are plenty of corner features so point correspondences should work. You might want to look into large baseline stereo (numerical stability?). Did you try different feature detectors/descriptors and different scale space settings? I'd try to detect 20-30 stable features, if possible.

Note, if you have tried the detailed_stitching demo of OpenCV only, it is assuming a fixed camera rotation. But your camera is translated and rotated.

Duffycola gravatar imageDuffycola ( 2014-07-21 10:36:51 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2014-07-20 06:08:07 -0500

Seen: 1,551 times

Last updated: Jul 20 '14