Ask Your Question
1

Line segment detection Near horizontal

asked 2015-04-05 17:12:50 -0600

HappyForce gravatar image

Hi,

I started my search on stackoverflow. As you may read in the given link I have several PDF's which I have now converted to JPG's in a decent format. With following script:

        "C:\Program Files\gs\gs9.15\bin\gswin64c.exe" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 -sDEVICE=jpeg -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r150 -sOutputFile=' ."$without_extension.jpg $_"

With perl for who is wondering.

So I have a bunch of JPG's which are like:

C:\fakepath\Opwekking0001.jpg

C:\fakepath\Opwekking0002.jpg

C:\fakepath\Opwekking0003.jpg

Most important part of these pictures is the thick black vertical and horizontal lines and the thick number which represents the number of the song.

The goal is to get the coordinates of the sections split by horizontal lines (and the pages are all in 2 columns).

I have researched and found that HoughP would be the solution. Correct me if I'm wrong please. So as test I constructed C++ code with OpenCV to recognize the horizontal and the vertical lines since I'll have to split on these. I have drawn an example on stackoverflow.

As extra I would like to detect the thick black number which points to the number of the song. But first things first. Detection of the lines is not going as expected:

Result:

C:\fakepath\LineDetection1.jpg

C:\fakepath\LineDetection2.jpg

C:\fakepath\LineDetection3.jpg

Same order as the originals. As you can see not all horizontal lines are given. And some are not completely till the end or they are broken up in pieces. Example 2 on the other hand is exactly what I want.

The code I have pasted on pastebin since this post is getting quite long.

http://pastebin.com/vxyLW6i3

I hope this is clear enough. I'll provide anything if you need more information. Maybe I'm even taking the wrong angle to getting the sections if you have a better idea please do share.

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
6

answered 2015-04-05 20:48:51 -0600

theodore gravatar image

updated 2015-04-14 16:21:48 -0600

Following more or less the technique that I am applying here and here. I am getting the following results, which it seems to be what you want.

image description

image description

image description

I consider that the title of the song has always the same distance from the detected horizontal line, if this is not the case then some further modification most likely will be needed. I am not uploading any code since most of it is based on the two links that I have provided earlier. However, if you still want it just tell me and I will uploaded here.


here you go:

#include <iostream>
#include <opencv2/opencv.hpp>

using namespace std;
using namespace cv;

int main()
{
    Mat src = imread("sheet1.jpg");

    if(!src.data)
    {
        cerr << "Problem loading image!!!" << endl;
        return EXIT_FAILURE;
    }

//        imshow("src", src);

    // resizing for practical reasons, small screen here :-p
    Mat rsz;
    Size size(800, 900);
    resize(src, rsz, size);

    imshow("rsz", rsz);

    Mat gray;
    cvtColor(rsz, gray, CV_BGR2GRAY);

    // Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
    Mat bw;
    adaptiveThreshold(~gray, bw, 255, CV_ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 15, -2);

    // Dilate a bit in order to correct possible gaps
    Mat kernel = Mat::ones(2, 2, CV_8UC1);
    dilate(bw, bw, kernel);

    // Show binary image
    imshow("bin", bw);

    // Create the images that will use to extract the horizonta and vertical lines
    Mat horizontal = bw.clone();

    // Specify size on horizontal axis
    int scale = 10; // play with this variable in order to increase/decrease the amount of lines to be detected
    int horizontalsize = horizontal.cols / scale;

    // Create structure element for extracting horizontal lines through morphology operations
    Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));

    // Apply morphology operations
    erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
    dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));

    // Show extracted horizontal lines
    imshow("horizontal", horizontal);


    // Create the images that will use to extract the horizonta and vertical lines
    Mat vertical = bw.clone();

    // Specify size on horizontal axis
    int verticalsize = vertical.cols / scale;

    // Create structure element for extracting horizontal lines through morphology operations
     Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));

    // Apply morphology operations
    erode(vertical, vertical, verticalStructure, Point(-1, -1));
    dilate(vertical, vertical, verticalStructure, Point(-1, -1));

    // Show extracted horizontal lines
    imshow("vertical", vertical);

    Mat joints = horizontal + vertical;
    imshow("joints", joints);

    // Find external contour
    vector<Vec4i> hierarchy;
    std::vector<std::vector<cv::Point> > contours;
    cv::findContours(horizontal, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));

    vector<vector<Point> > contours_poly( contours.size() );
    vector<Rect> boundRect( contours.size() );

    // Draw the contour as a solid blob filling also any convexity defect with the extracted hulls
    for (size_t i = 0; i < contours.size(); i++)
    {
//        cout << boundRect[i].tl() << endl;
//        cout << boundRect[i].br() << endl << endl;

//        cout << arcLength(cv::Mat(contours[i]), true) << endl;
        double length = arcLength(cv::Mat(contours[i]), true);

        // filter long and short lines
        if(length < 75 || length > 1500)
            continue;

        approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );
        boundRect[i] = boundingRect( Mat(contours_poly[i]) );
//        if(length > 200)
//        {
//            boundRect[i] += Size(0, -40); // expanding rectangle by a certain amount
//            boundRect[i] -= Point(0, 3); // shifting rectangle by a certain offset ...
(more)
edit flag offensive delete link more

Comments

I am trying to do it myself with the methods you provided. Am I correct that I should be able to find the lines with this link and the song numbers with this link

Or do I need to combine them for both? ---

EDIT:

Well I'm making progress I got the horizontal and vertical lines with the first method as mentioned and now I see how I can use the second method to draw the lines you have shown in your example !! I like this alot !!!

HappyForce gravatar imageHappyForce ( 2015-04-08 12:19:25 -0600 )edit

Actually the second link make use of the method in the first link plus some extra code. To be honest for the images in your case I have made some little modifications as well but not that many and there are quite easy to be done for someone with some previous experience. I think that it is nice that you are trying to understand and implement it by yourself, however as I told you when you fill that you stacked somewhere or you cannot understand something just ask here. I will try to help, moreover whenever you want I can provide you access to the full code.

theodore gravatar imagetheodore ( 2015-04-08 14:00:35 -0600 )edit

Theodore, I will ask for the full code now. I have written code and it works mostly but I would like to compare to yours. Especially since I have some places where my code fails. (e.g. too many horizontal rows)

HappyForce gravatar imageHappyForce ( 2015-04-14 13:43:41 -0600 )edit
2

@HappyForce I updated my answer, now you can find the code as well.

theodore gravatar imagetheodore ( 2015-04-14 16:22:46 -0600 )edit
2

halleluja ! ;)

berak gravatar imageberak ( 2015-04-15 03:15:55 -0600 )edit

@theodore, seriously you got to many spare time. Once I finish the book I am working on, we should discuss releasing a application booklet based on all your entries here :D

StevenPuttemans gravatar imageStevenPuttemans ( 2015-06-07 04:13:56 -0600 )edit
1

@StevenPuttemans that's true, indeed at the moment I have some time to spare for helping people here or elsewhere. Nevertheless, I find quite interesting some of the questions from the users, plus I think that it is a mutual interaction, since sometimes it helps me as well to keep up to date with the library and at the same time obtain experience in different subjects regarding image processing/computer vision and opencv (it is really amazing sometimes what people ask and what they want to implement using computer vision). However, soon I think that I will not have some much time. Of course, this will not prevent me from being around :-p. As for the booklet, as I told you before I am totally in I guess other users would like to join as well. So whenever, you feel ready just heat me since

theodore gravatar imagetheodore ( 2015-06-07 06:20:34 -0600 )edit

it seems that you posses some experience regarding that and how to do it, which I do not :-).

theodore gravatar imagetheodore ( 2015-06-07 06:22:30 -0600 )edit
1

@theodore, I simply like your enthousiasme and thinking. More people like that would help grow this place alot. It is basically the whole idea behind an open source community though many people do not grasp that ;)

StevenPuttemans gravatar imageStevenPuttemans ( 2015-06-07 07:04:04 -0600 )edit

@StevenPuttemans, thanks ;-) and I totally agree with what you are saying about the open source community and the whole idea behind it.

theodore gravatar imagetheodore ( 2015-06-07 07:45:00 -0600 )edit
0

answered 2015-06-04 10:27:21 -0600

theodore gravatar image

updated 2015-06-05 20:43:49 -0600

Hi @HappyForce2 the following code will do the job:

Mat roi;
threshold(gray(boundRect[i]), roi, 100, 255, CV_THRESH_BINARY/* | CV_THRESH_OTSU*/);

if(countNonZero(~roi) > 0)
    cout << "There is a No. Song!!!" << endl;
else
    cout << "Title is empty!!!" << endl;

just add it into the for loop that you are saying and before the following code:

    drawContours( rsz, contours, i, Scalar(0, 0, 255), CV_FILLED, 8, vector<Vec4i>(), 0, Point() );
    rectangle( rsz, boundRect[i].tl(), boundRect[i].br(), Scalar(0, 255, 0), 1, 8, 0 );
}

Here works pretty much ok, if you have any question or face any issue just tell me. Enjoy ;-)

edit flag offensive delete link more

Comments

Strange... locally when I use your basic code you have posted up there and put this code in it it breaks... with following error:

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= oi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat, file C:\builds\master_PackSlave-in32-vc12-shared\opencv\modules\core\src\matrix.cpp, line 492
HappyForce2 gravatar imageHappyForce2 ( 2015-06-05 16:43:10 -0600 )edit

yup that's due to some false lines/noise that you get at the bottom of the paper. Try to add, this code:

// filter lines at the bottom of the page
if(boundRect[i].y > rsz.rows - 10)
    continue;

after the following lines of code:

approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );
boundRect[i] = boundingRect( Mat(contours_poly[i]) );

and then it should work.

theodore gravatar imagetheodore ( 2015-06-05 20:43:04 -0600 )edit
0

answered 2015-05-30 07:58:34 -0600

@theodore Hi (i'm happyforce but I was not able to log in with my previous account since it used (google button))

I have just 1 question left my program is nearly finished I only need to know 1 thing.

I have the rectangles which would cover the song numbers (green in the picture you showed) . I need to be able to check if there is something in the box. As it happens sometimes there are lines too much at the end of the pages. For example:

image description : you can see at the end of a column we have a line and at the start of the next column we have another line. I need to be able to filter these situations so I was thinking I can checke whether the song number is there or not.

So in short:

I need a function/ functions to check if a rectangle is empty on the given file.

As it seems simple I have not found an example yet.

edit flag offensive delete link more

Comments

@HappyForce2 just try to apply the findContours() function into the ROI represented from the rectangle. If there are contours, then theres is a song number if not then there isn't. If I find some time I will try to upload an example. Or if you want to obtain the letters and numbers of the song title then you will need to apply an OCR algorithm. For the latter you can have a look in this example or search in the web there are some really nice tut's and examples.

theodore gravatar imagetheodore ( 2015-05-31 14:49:56 -0600 )edit

I thought next piece of code was going to do the trick :( ... but it gives exception ...

vector<Vec4i> rectangleHierarchy;
std::vector<std::vector<cv::Point> > rectangleContours;
Mat roi(rsz, boundRect[i]);
cv::findContours(roi, rectangleContours, rectangleHierarchy, cv::RETR_CCOMP, cv::CHAIN_APPROX_SIMPLE, Point(0, 0));
if (rectangleContours.size() != 0){
    cout << "Rectangle has no song number!" << roi << endl;
}
else{
    cout << "Ah Rectangle" << roi << endl;
}

Which I will put at the end of this for:

for (size_t i = 0; i < contours.size(); i++)
{
    //        cout << boundRect[i].tl() << endl;
    //        cout << boundRect[i].br() << endl << endl;

    //        cout << arcLength(cv::Mat(contours[i]), true) << endl;
    double length = arcLength(cv::Mat(contours[i]), true
HappyForce2 gravatar imageHappyForce2 ( 2015-06-03 14:12:47 -0600 )edit

Question Tools

3 followers

Stats

Asked: 2015-04-05 17:12:50 -0600

Seen: 3,646 times

Last updated: Jun 05 '15