Ask Your Question

How to merge nearby rectangles

asked 2018-11-26 23:21:19 -0500

Rex Low gravatar image

updated 2018-12-02 05:58:55 -0500

I am working on character segmentation and looking to merge nearby characters into a box. I have successfully found the contours but I am not sure how to find a straight line between the top and bottom points.

def findContours(self, image):
    contour_img = image.copy()
    vis = image.copy()
    vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
    _, contours, hierarchy = cv2.findContours(contour_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    contoursBBS = []
    for contour in contours:
        [x, y, w, h] = cv2.boundingRect(contour)

        if w > 100 or h < 8:

        contoursBBS.append([x, y, w, h, w*h])
    contoursBBS = np.array(contoursBBS)

    # reject outliers based on width, height, and area
    rejectH = self.rejectOutliers(contoursBBS, 3, m=4)
    rejectW = self.rejectOutliers(rejectH, 2, m=4)
    rejectA = self.rejectOutliers(rejectW, 4, m=4)

    contourRects = []
    for c in rejectA:
        [x, y, w, h, a] = c
        if w < 9 or h < 15 or a < 300 or a > 6000:

    for i, rect in enumerate(contourRects):
        [x, y, w, h, a] = rect
        topCenter, bottomCenter = (int((2*x + w)/2), int(y)), (int((2*x + w)/2), int(y+h))

        print("X: {:4d}  Y:  {:4d}  W: {:4d}  H: {:4d}  A: {:4d}".format(x, y, w, h, a))

        cv2.rectangle(vis, (x, y), (x+w, y+h), (0, 255, 0), 2), topCenter, 2, (0, 0, 255), 2), bottomCenter, 2, (0, 0, 255), 2)


image description

What I am hoping to achieve

image description

Some test images (requested by @supra56)

  4. (very challenging)
edit retag flag offensive close merge delete

3 answers

Sort by » oldest newest most voted

answered 2018-11-26 23:33:55 -0500

berak gravatar image

untested, but you could add all the contour points to a single array, and call minAreaRect on it:

arr = []
for x,y,w,h in contourRects:

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners
edit flag offensive delete link more


I tried but it computes the average rectangles, like this (the blue rectangle)

Rex Low gravatar imageRex Low ( 2018-11-26 23:42:50 -0500 )edit

you seem to do that with your "center" points, not with the actual contour ones. you also have to get rid of the 2 outlier rects at the left / right side first

berak gravatar imageberak ( 2018-11-26 23:45:57 -0500 )edit

I tried again with actual contours ones and the results are better. Can you show me how to get rid of the outliers?

Rex Low gravatar imageRex Low ( 2018-11-26 23:51:35 -0500 )edit

maybe filter for large height ?

berak gravatar imageberak ( 2018-11-27 00:00:12 -0500 )edit

characters have similar size, how about pick median value and filter by a multiplemedian e.g. (height > 0.75median && height < 1.25*median)

blues gravatar imageblues ( 2018-11-27 00:25:00 -0500 )edit

@berak My application will process real time images of different heights, so I’ll have to figure out a dynamic algorithm for an optimal height. @blues I’ll try this later. Thanks!

Rex Low gravatar imageRex Low ( 2018-11-27 00:27:50 -0500 )edit

btw, we do have proper text detection methods builtin.

berak gravatar imageberak ( 2018-11-27 00:36:40 -0500 )edit

Thanks for the information. But I found that EAST will introduce extra computational costs.

Rex Low gravatar imageRex Low ( 2018-11-27 00:41:31 -0500 )edit

@Rex Low. Can you post orginal image?

supra56 gravatar imagesupra56 ( 2018-11-29 05:41:51 -0500 )edit

@supra56 Hi, sorry for late response. I have updated the post with several test images. Please have a look at them :)

Rex Low gravatar imageRex Low ( 2018-12-02 05:59:33 -0500 )edit

answered 2020-03-04 03:17:46 -0500

wilsonchoo gravatar image

Append all your rectangles in rectList and use cv2.boundingRect() to get the bigger rectangle that enclose all sub rectangles. Let me know if it works for you. Thanks berak's for the piece of code

arr = []
for x,y,w,h in rectList:

x,y,w,h = cv2.boundingRect(np.asarray(arr))
edit flag offensive delete link more

answered 2020-08-13 22:51:08 -0500

jcrist gravatar image

I found better solution without iterating through contours - by using numpy concatenate() function:

concat = np.concatenate(contours)
hulls = cv2.convexHull(concat)

Resulting hulls contour can be further approximated by cv2.approxPolyDP() function in order to get bounding rectangle.

edit flag offensive delete link more

Question Tools

1 follower


Asked: 2018-11-26 23:21:19 -0500

Seen: 7,652 times

Last updated: Aug 13 '20