How to merge nearby rectangles

asked 2018-11-26 23:21:19 -0600

Rex Low
26 ●1 ●2 ●5

updated 2018-12-02 05:58:55 -0600

I am working on character segmentation and looking to merge nearby characters into a box. I have successfully found the contours but I am not sure how to find a straight line between the top and bottom points.

def findContours(self, image):
    contour_img = image.copy()
    vis = image.copy()
    vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
    _, contours, hierarchy = cv2.findContours(contour_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    contoursBBS = []
    for contour in contours:
        [x, y, w, h] = cv2.boundingRect(contour)

        if w > 100 or h < 8:
            continue

        contoursBBS.append([x, y, w, h, w*h])
    contoursBBS = np.array(contoursBBS)

    # reject outliers based on width, height, and area
    rejectH = self.rejectOutliers(contoursBBS, 3, m=4)
    rejectW = self.rejectOutliers(rejectH, 2, m=4)
    rejectA = self.rejectOutliers(rejectW, 4, m=4)

    contourRects = []
    for c in rejectA:
        [x, y, w, h, a] = c
        if w < 9 or h < 15 or a < 300 or a > 6000:
            continue
        contourRects.append(c)

    for i, rect in enumerate(contourRects):
        [x, y, w, h, a] = rect
        topCenter, bottomCenter = (int((2*x + w)/2), int(y)), (int((2*x + w)/2), int(y+h))

        print("X: {:4d}  Y:  {:4d}  W: {:4d}  H: {:4d}  A: {:4d}".format(x, y, w, h, a))


        cv2.rectangle(vis, (x, y), (x+w, y+h), (0, 255, 0), 2)
        cv2.circle(vis, topCenter, 2, (0, 0, 255), 2)
        cv2.circle(vis, bottomCenter, 2, (0, 0, 255), 2)

Result

image description

What I am hoping to achieve

image description

Some test images (requested by @supra56)

https://ibb.co/bgtyypp
https://ibb.co/XtdHgWg
https://ibb.co/5R4QVTR
https://ibb.co/gD8KymR (very challenging)
https://ibb.co/k5YDT1Q

edit retag flag offensive close merge delete

add a comment

3 answers

Sort by » oldest newest most voted

answered 2018-11-26 23:33:55 -0600

berak
32993 ●7 ●81 ●312

untested, but you could add all the contour points to a single array, and call minAreaRect on it:

arr = []
for x,y,w,h in contourRects:
      arr.append((x,y))
      arr.append((x+w,y+h))

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners

edit flag offensive delete link

Comments

I tried but it computes the average rectangles, like this (the blue rectangle) https://ibb.co/H2JjD1B

Rex Low ( 2018-11-26 23:42:50 -0600 )edit

you seem to do that with your "center" points, not with the actual contour ones. you also have to get rid of the 2 outlier rects at the left / right side first

berak ( 2018-11-26 23:45:57 -0600 )edit

I tried again with actual contours ones and the results are better. Can you show me how to get rid of the outliers?

Rex Low ( 2018-11-26 23:51:35 -0600 )edit

maybe filter for large height ?

berak ( 2018-11-27 00:00:12 -0600 )edit

characters have similar size, how about pick median value and filter by a multiplemedian e.g. (height > 0.75median && height < 1.25*median)

blues ( 2018-11-27 00:25:00 -0600 )edit

@berak My application will process real time images of different heights, so I’ll have to figure out a dynamic algorithm for an optimal height. @blues I’ll try this later. Thanks!

Rex Low ( 2018-11-27 00:27:50 -0600 )edit

btw, we do have proper text detection methods builtin.

berak ( 2018-11-27 00:36:40 -0600 )edit

Thanks for the information. But I found that EAST will introduce extra computational costs.

Rex Low ( 2018-11-27 00:41:31 -0600 )edit

@Rex Low. Can you post orginal image?

supra56 ( 2018-11-29 05:41:51 -0600 )edit

@supra56 Hi, sorry for late response. I have updated the post with several test images. Please have a look at them :)

Rex Low ( 2018-12-02 05:59:33 -0600 )edit

see more comments

answered 2020-03-04 03:17:46 -0600

wilsonchoo
11 ●1

Append all your rectangles in rectList and use cv2.boundingRect() to get the bigger rectangle that enclose all sub rectangles. Let me know if it works for you. Thanks berak's for the piece of code


arr = []
for x,y,w,h in rectList:
    arr.append((x,y))
    arr.append((x+w,y+h))

x,y,w,h = cv2.boundingRect(np.asarray(arr))
cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),1)

edit flag offensive delete link

add a comment

answered 2020-08-13 22:51:08 -0600

jcrist
156 ●1 ●10

I found better solution without iterating through contours - by using numpy concatenate() function:

concat = np.concatenate(contours)
hulls = cv2.convexHull(concat)

Resulting hulls contour can be further approximated by cv2.approxPolyDP() function in order to get bounding rectangle.

edit flag offensive delete link

add a comment

How to merge nearby rectangles

3 answers

Comments

Links

Question Tools

Stats

Related questions

How to merge nearby rectangles edit

3 answers

Comments

Links

Question Tools

Stats

Related questions

How to merge nearby rectangles