Ask Your Question

How to sort contours left to right, while going top to bottom

asked 2016-07-29 09:16:59 -0500

AkashShrestha gravatar image

updated 2017-09-26 02:30:20 -0500

I'm finding the contours for an image with digits and characters, for OCR. So, I need the contours to be sorted left to right, while going line to line, i.e. top to bottom. Right now, the contours aren't sorted that way.

image description

For example, the contours for the above image is sorted randomly.

What I need is the sorting as D,o,y,o,u,k,n,o,w,s,o,m,e,o,n,e,r,.(dot),i(without dot),c,h...and so on. I've tried couple of methods where we first observe the y-coordinate and then use some keys and the x-coordinate. Like right now, I have the following sorting code. It works for the first 2 lines. Then in the 3rd line, the sorting somehow doesn't happen. The main problem seem to be in the letters such as i, j, ?, (dot), (comma), etc where the y axis of the (dot) varies, despite belonging to the same line. So what might be a good solution for this?

for ctr in contours:    
    if cv2.contourArea(ctr) > maxArea * areaRatio: 

for i in rect:
    x = i[0]
    y = i[1]
    w = i[2]
    h = i[3]

        max_line_height = h

mlh = max_line_height*2
max_line_width = raw_image.shape[1] #width of the input image
mlw = max_line_width
rect = np.asarray(rect)
s = rect.astype( np.uint32 ) #prevent overflows
order= mlw*(s[:,1]/mlh)+s[:,0]
sort_order= np.argsort( order )
rect = rect[ sort_order ]
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2016-08-01 09:22:51 -0500

TommyLi gravatar image

Firstly you need to use boundingRect() to generate Rect, which will give you coordination. (and looks like you did.)

Then you can use sorted() with a key function to generate key value using (x, y), something like key = y * 3000 + x.

This should do the work.

edit flag offensive delete link more


Could you please provide a code snippet for what you've just described? Also will that work for sorting alphabet contours as well? They have different y values for taller and smaller characters.

AkashShrestha gravatar imageAkashShrestha ( 2016-08-05 05:35:16 -0500 )edit

answered 2016-08-08 07:17:44 -0500

essamzaky gravatar image

updated 2016-08-08 07:20:24 -0500

The problem you are looking for called "Page layout detection and character segmentation" , the generic steps go as follow:

  1. Detect page zones such as , Text Headers , Text paragraph , Graphics and pictures , tables , ....
  2. For Text zones (Header , table cell , paragraph) do the following.
  3. Split into lines
  4. split line into words
  5. split word into characters

In your case you only have one paragraph -you can split paragraph by using horizontal histogram and cut line on local minimum , or you can use contours by adding regions which share vertically some height threshold into one line.

  • -sort the lines from top to bottom.
  • -for every line sort regions from left to right.
  • -In one line if there is horizontal overlap , merge the two regions in one bigger region (this will solve i,j problems)
  • -Then you can split the line into characters by taking every region as one character or ligature (rr,ff,vv).

finally if you need ready made solution , Tesseract can do all previous tasks plus the recognition

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2016-07-29 09:16:01 -0500

Seen: 5,981 times

Last updated: Aug 08 '16