Ask Your Question
0

Contour perspective warp

asked 2016-04-02 11:46:22 -0500

TimK gravatar image

updated 2016-04-03 04:32:22 -0500

I want to automate a process where text is recognized from a specific card type:

image description

I will first explain what I dit so far. First I converted the image to a grayscale one:

image description

Then I applied Canny:

image description

By the result of this, it was easy to find the largest contour:

image description

So far so good. The problem is: I have a shape here which fits in rectangle. However, the perspective is not top-down. Which implies that it is not possible to use boundingRect. Basically I'm in need of an algorithm to find the orientation of this contour.

I have no idea how to do this. Is there something I am missing? How would you do this?

Edit: Note that the above photo is not a good example, let me add another test case:

image description

edit retag flag offensive close merge delete

Comments

You can use moment to rotate your shape using your largest contour

LBerger gravatar imageLBerger ( 2016-04-02 11:56:59 -0500 )edit

@LBerger How would I do that?

TimK gravatar imageTimK ( 2016-04-02 12:01:21 -0500 )edit

you can use Moment or may be minAreaRect

LBerger gravatar imageLBerger ( 2016-04-02 14:27:24 -0500 )edit

@TimK except to the solution from @LBerger you can have a look in this example here, though I think it is more or less based on the concept that @LBerger suggested

theodore gravatar imagetheodore ( 2016-04-02 17:01:48 -0500 )edit

@theodore That only works on pictures with a top down perspective. I've added a new photo (see edit), to illustrate what I mean and hopefully clear things up! Thanks for you comment.

TimK gravatar imageTimK ( 2016-04-03 04:39:38 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2016-04-04 07:23:17 -0500

You need to find first the Homography. Have a look at this tutorial. Then, it would be easier to reverse the usage of scene points and object points to get the transformation that gives you the rectangle in the way you want (instead of finding the orientation of the rectangle in the scene, you try to find the fronto-parallel orientation, if that does make sense).

You could also have a look at the affine transformation estimation made by the estimageRigidTransform function or the findTransformEEC function.

edit flag offensive delete link more
1

answered 2016-04-02 15:25:19 -0500

LBerger gravatar image

updated 2016-04-03 05:47:32 -0500

May be something like this

int main(int argc, char* argv[])
{
    VideoCapture v(0);
    Mat im=imread("14596150505143125.jpg");
    Mat img,dst;
    double thresh=0;
    vector<vector <Point>> ctr;
    cvtColor(im,img,CV_BGR2GRAY);
    threshold(img,dst,thresh,255,THRESH_OTSU);
    findContours(dst,ctr,CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);

    int idx,maxCtr=-1;
    double pi = acos(-1.0);
    for (int i = 0; i<ctr.size();i++)
        if (saturate_cast<int>(ctr[i].size())>maxCtr)
        {
            maxCtr=ctr[i].size();
            idx=i;
        }
    RotatedRect r=minAreaRect(ctr[idx]);
    cout<<r.center<<"\t Size ="<<r.size<<"\tAngle="<<r.angle<<"\n";
    Mat rt1=(Mat_<float>(3,3)<< 1,0,-r.center.x,0,1,-r.center.y,0,0,1);
    double theta=(270-r.angle)/180*pi;
    Mat rr=(Mat_<float>(3,3)<< cos(theta),-sin(theta),0,sin(theta),cos(theta),0,0,0,1);
    Mat rt2=(Mat_<float>(3,3)<< 1,0,r.center.x,0,1,r.center.y,0,0,1);
    Mat rAffine;
    Mat rt=(rt2*rr*rt1);
    rt(Rect(0,0,3,2)).copyTo(rAffine);
    Mat imR;
    cout<<rAffine;
    warpAffine(im, imR,rAffine, im.size());
    imshow("result",imR);
    waitKey();
    return 0;
}

If there is a perpesctive you can use this program. You have to check if point in ctrApprox and rotatedrect are in same order

#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/videoio/videoio.hpp"
#include "opencv2/calib3d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/highgui.hpp"

#include <iostream>
#include <fstream>
#include <algorithm>
#include <ctype.h>

using namespace cv;
using namespace std;



int main(int argc, char* argv[])
{
    VideoCapture v(0);
    Mat im=imread("C:/Users/Laurent.PC-LAURENT-VISI/Downloads/14596150505143125.jpg");
    Mat img,dst;
    double thresh=0;
    vector<vector <Point>> ctr;
    cvtColor(im,img,CV_BGR2GRAY);
    threshold(img,dst,thresh,255,THRESH_OTSU);
    findContours(dst,ctr,CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);

    int idx,maxCtr=-1;
    double pi = acos(-1.0);
    for (int i = 0; i<ctr.size();i++)
        if (saturate_cast<int>(ctr[i].size())>maxCtr)
        {
            maxCtr=ctr[i].size();
            idx=i;
        }
    vector<Point> ctrApprox;
    double epsilon=2;
    do
    {
        approxPolyDP(ctr[idx], ctrApprox, epsilon, true);
        epsilon++;    
    }
    while (ctrApprox.size()>4);
    RotatedRect r=minAreaRect(ctr[idx]);
    vector <Point2f> p2(4);
    p2[0] = Point2f(r.center.x - r.size.width/2,r.center.x - r.size.height/2);
    p2[1] = Point2f(r.center.x - r.size.width/2,r.center.x + r.size.height/2);
    p2[2] = Point2f(r.center.x + r.size.width/2,r.center.x + r.size.height/2);
    p2[3] = Point2f(r.center.x + r.size.width/2,r.center.x - r.size.height/2);
    Mat rth=findHomography(ctrApprox,p2);
    warpPerspective(im, imR,rth, Size(im.size().width*2,im.size().height*2));
    imshow("result",imR);
    waitKey();
    return 0;
}
edit flag offensive delete link more

Comments

I appreciate your answer. I might have not been clear enough, but I need it to work with any perspective. Please take a look at the last photo in my question (which I edited in).

TimK gravatar imageTimK ( 2016-04-03 04:36:47 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2016-04-02 11:46:22 -0500

Seen: 1,635 times

Last updated: Apr 04 '16