# Line segment detection Near horizontal

Hi,

I started my search on stackoverflow. As you may read in the given link I have several PDF's which I have now converted to JPG's in a decent format. With following script:

"C:\Program Files\gs\gs9.15\bin\gswin64c.exe" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 -sDEVICE=jpeg -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r150 -sOutputFile=' ."$without_extension.jpg$_"

With perl for who is wondering.

So I have a bunch of JPG's which are like:

C:\fakepath\Opwekking0001.jpg

C:\fakepath\Opwekking0002.jpg

C:\fakepath\Opwekking0003.jpg

Most important part of these pictures is the thick black vertical and horizontal lines and the thick number which represents the number of the song.

The goal is to get the coordinates of the sections split by horizontal lines (and the pages are all in 2 columns).

I have researched and found that HoughP would be the solution. Correct me if I'm wrong please. So as test I constructed C++ code with OpenCV to recognize the horizontal and the vertical lines since I'll have to split on these. I have drawn an example on stackoverflow.

As extra I would like to detect the thick black number which points to the number of the song. But first things first. Detection of the lines is not going as expected:

Result:

C:\fakepath\LineDetection1.jpg

C:\fakepath\LineDetection2.jpg

C:\fakepath\LineDetection3.jpg

Same order as the originals. As you can see not all horizontal lines are given. And some are not completely till the end or they are broken up in pieces. Example 2 on the other hand is exactly what I want.

The code I have pasted on pastebin since this post is getting quite long.

http://pastebin.com/vxyLW6i3

I hope this is clear enough. I'll provide anything if you need more information. Maybe I'm even taking the wrong angle to getting the sections if you have a better idea please do share.

edit retag close merge delete

Sort by » oldest newest most voted

Following more or less the technique that I am applying here and here. I am getting the following results, which it seems to be what you want.

I consider that the title of the song has always the same distance from the detected horizontal line, if this is not the case then some further modification most likely will be needed. I am not uploading any code since most of it is based on the two links that I have provided earlier. However, if you still want it just tell me and I will uploaded here.

here you go:

#include <iostream>
#include <opencv2/opencv.hpp>

using namespace std;
using namespace cv;

int main()
{

if(!src.data)
{
return EXIT_FAILURE;
}

//        imshow("src", src);

// resizing for practical reasons, small screen here :-p
Mat rsz;
Size size(800, 900);
resize(src, rsz, size);

imshow("rsz", rsz);

Mat gray;
cvtColor(rsz, gray, CV_BGR2GRAY);

// Apply adaptiveThreshold at the bitwise_not of gray, notice the ~ symbol
Mat bw;

// Dilate a bit in order to correct possible gaps
Mat kernel = Mat::ones(2, 2, CV_8UC1);
dilate(bw, bw, kernel);

// Show binary image
imshow("bin", bw);

// Create the images that will use to extract the horizonta and vertical lines
Mat horizontal = bw.clone();

// Specify size on horizontal axis
int scale = 10; // play with this variable in order to increase/decrease the amount of lines to be detected
int horizontalsize = horizontal.cols / scale;

// Create structure element for extracting horizontal lines through morphology operations
Mat horizontalStructure = getStructuringElement(MORPH_RECT, Size(horizontalsize,1));

// Apply morphology operations
erode(horizontal, horizontal, horizontalStructure, Point(-1, -1));
dilate(horizontal, horizontal, horizontalStructure, Point(-1, -1));

// Show extracted horizontal lines
imshow("horizontal", horizontal);

// Create the images that will use to extract the horizonta and vertical lines
Mat vertical = bw.clone();

// Specify size on horizontal axis
int verticalsize = vertical.cols / scale;

// Create structure element for extracting horizontal lines through morphology operations
Mat verticalStructure = getStructuringElement(MORPH_RECT, Size( 1,verticalsize));

// Apply morphology operations
erode(vertical, vertical, verticalStructure, Point(-1, -1));
dilate(vertical, vertical, verticalStructure, Point(-1, -1));

// Show extracted horizontal lines
imshow("vertical", vertical);

Mat joints = horizontal + vertical;
imshow("joints", joints);

// Find external contour
vector<Vec4i> hierarchy;
std::vector<std::vector<cv::Point> > contours;
cv::findContours(horizontal, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));

vector<vector<Point> > contours_poly( contours.size() );
vector<Rect> boundRect( contours.size() );

// Draw the contour as a solid blob filling also any convexity defect with the extracted hulls
for (size_t i = 0; i < contours.size(); i++)
{
//        cout << boundRect[i].tl() << endl;
//        cout << boundRect[i].br() << endl << endl;

//        cout << arcLength(cv::Mat(contours[i]), true) << endl;
double length = arcLength(cv::Mat(contours[i]), true);

// filter long and short lines
if(length < 75 || length > 1500)
continue;

approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );
boundRect[i] = boundingRect( Mat(contours_poly[i]) );
//        if(length > 200)
//        {
//            boundRect[i] += Size(0, -40); // expanding rectangle by a certain amount
//            boundRect[i] -= Point(0, 3); // shifting rectangle by a certain offset ...
more

I am trying to do it myself with the methods you provided. Am I correct that I should be able to find the lines with this link and the song numbers with this link

Or do I need to combine them for both? ---

EDIT:

Well I'm making progress I got the horizontal and vertical lines with the first method as mentioned and now I see how I can use the second method to draw the lines you have shown in your example !! I like this alot !!!

( 2015-04-08 12:19:25 -0600 )edit

Actually the second link make use of the method in the first link plus some extra code. To be honest for the images in your case I have made some little modifications as well but not that many and there are quite easy to be done for someone with some previous experience. I think that it is nice that you are trying to understand and implement it by yourself, however as I told you when you fill that you stacked somewhere or you cannot understand something just ask here. I will try to help, moreover whenever you want I can provide you access to the full code.

( 2015-04-08 14:00:35 -0600 )edit

Theodore, I will ask for the full code now. I have written code and it works mostly but I would like to compare to yours. Especially since I have some places where my code fails. (e.g. too many horizontal rows)

( 2015-04-14 13:43:41 -0600 )edit
2

@HappyForce I updated my answer, now you can find the code as well.

( 2015-04-14 16:22:46 -0600 )edit
2

halleluja ! ;)

( 2015-04-15 03:15:55 -0600 )edit

@theodore, seriously you got to many spare time. Once I finish the book I am working on, we should discuss releasing a application booklet based on all your entries here :D

( 2015-06-07 04:13:56 -0600 )edit
1

@StevenPuttemans that's true, indeed at the moment I have some time to spare for helping people here or elsewhere. Nevertheless, I find quite interesting some of the questions from the users, plus I think that it is a mutual interaction, since sometimes it helps me as well to keep up to date with the library and at the same time obtain experience in different subjects regarding image processing/computer vision and opencv (it is really amazing sometimes what people ask and what they want to implement using computer vision). However, soon I think that I will not have some much time. Of course, this will not prevent me from being around :-p. As for the booklet, as I told you before I am totally in I guess other users would like to join as well. So whenever, you feel ready just heat me since

( 2015-06-07 06:20:34 -0600 )edit

it seems that you posses some experience regarding that and how to do it, which I do not :-).

( 2015-06-07 06:22:30 -0600 )edit
1

@theodore, I simply like your enthousiasme and thinking. More people like that would help grow this place alot. It is basically the whole idea behind an open source community though many people do not grasp that ;)

( 2015-06-07 07:04:04 -0600 )edit

@StevenPuttemans, thanks ;-) and I totally agree with what you are saying about the open source community and the whole idea behind it.

( 2015-06-07 07:45:00 -0600 )edit

Hi @HappyForce2 the following code will do the job:

Mat roi;
threshold(gray(boundRect[i]), roi, 100, 255, CV_THRESH_BINARY/* | CV_THRESH_OTSU*/);

if(countNonZero(~roi) > 0)
cout << "There is a No. Song!!!" << endl;
else
cout << "Title is empty!!!" << endl;

just add it into the for loop that you are saying and before the following code:

drawContours( rsz, contours, i, Scalar(0, 0, 255), CV_FILLED, 8, vector<Vec4i>(), 0, Point() );
rectangle( rsz, boundRect[i].tl(), boundRect[i].br(), Scalar(0, 255, 0), 1, 8, 0 );
}

Here works pretty much ok, if you have any question or face any issue just tell me. Enjoy ;-)

more

Strange... locally when I use your basic code you have posted up there and put this code in it it breaks... with following error:

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= oi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat, file C:\builds\master_PackSlave-in32-vc12-shared\opencv\modules\core\src\matrix.cpp, line 492
( 2015-06-05 16:43:10 -0600 )edit

yup that's due to some false lines/noise that you get at the bottom of the paper. Try to add, this code:

// filter lines at the bottom of the page
if(boundRect[i].y > rsz.rows - 10)
continue;

after the following lines of code:

approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );
boundRect[i] = boundingRect( Mat(contours_poly[i]) );

and then it should work.

( 2015-06-05 20:43:04 -0600 )edit

@theodore Hi (i'm happyforce but I was not able to log in with my previous account since it used (google button))

I have just 1 question left my program is nearly finished I only need to know 1 thing.

I have the rectangles which would cover the song numbers (green in the picture you showed) . I need to be able to check if there is something in the box. As it happens sometimes there are lines too much at the end of the pages. For example:

: you can see at the end of a column we have a line and at the start of the next column we have another line. I need to be able to filter these situations so I was thinking I can checke whether the song number is there or not.

So in short:

I need a function/ functions to check if a rectangle is empty on the given file.

more

@HappyForce2 just try to apply the findContours() function into the ROI represented from the rectangle. If there are contours, then theres is a song number if not then there isn't. If I find some time I will try to upload an example. Or if you want to obtain the letters and numbers of the song title then you will need to apply an OCR algorithm. For the latter you can have a look in this example or search in the web there are some really nice tut's and examples.

( 2015-05-31 14:49:56 -0600 )edit

I thought next piece of code was going to do the trick :( ... but it gives exception ...

vector<Vec4i> rectangleHierarchy;
std::vector<std::vector<cv::Point> > rectangleContours;
Mat roi(rsz, boundRect[i]);
cv::findContours(roi, rectangleContours, rectangleHierarchy, cv::RETR_CCOMP, cv::CHAIN_APPROX_SIMPLE, Point(0, 0));
if (rectangleContours.size() != 0){
cout << "Rectangle has no song number!" << roi << endl;
}
else{
cout << "Ah Rectangle" << roi << endl;
}

Which I will put at the end of this for:

for (size_t i = 0; i < contours.size(); i++)
{
//        cout << boundRect[i].tl() << endl;
//        cout << boundRect[i].br() << endl << endl;

//        cout << arcLength(cv::Mat(contours[i]), true) << endl;
double length = arcLength(cv::Mat(contours[i]), true
( 2015-06-03 14:12:47 -0600 )edit

Official site

GitHub

Wiki

Documentation