# How to merge nearby rectangles

I am working on character segmentation and looking to merge nearby characters into a box. I have successfully found the contours but I am not sure how to find a straight line between the top and bottom points.

def findContours(self, image):
contour_img = image.copy()
vis = image.copy()
vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
_, contours, hierarchy = cv2.findContours(contour_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

contoursBBS = []
for contour in contours:
[x, y, w, h] = cv2.boundingRect(contour)

if w > 100 or h < 8:
continue

contoursBBS.append([x, y, w, h, w*h])
contoursBBS = np.array(contoursBBS)

# reject outliers based on width, height, and area
rejectH = self.rejectOutliers(contoursBBS, 3, m=4)
rejectW = self.rejectOutliers(rejectH, 2, m=4)
rejectA = self.rejectOutliers(rejectW, 4, m=4)

contourRects = []
for c in rejectA:
[x, y, w, h, a] = c
if w < 9 or h < 15 or a < 300 or a > 6000:
continue
contourRects.append(c)

for i, rect in enumerate(contourRects):
[x, y, w, h, a] = rect
topCenter, bottomCenter = (int((2*x + w)/2), int(y)), (int((2*x + w)/2), int(y+h))

print("X: {:4d}  Y:  {:4d}  W: {:4d}  H: {:4d}  A: {:4d}".format(x, y, w, h, a))

cv2.rectangle(vis, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.circle(vis, topCenter, 2, (0, 0, 255), 2)
cv2.circle(vis, bottomCenter, 2, (0, 0, 255), 2)


Result

What I am hoping to achieve

Some test images (requested by @supra56)

edit retag close merge delete

Sort by » oldest newest most voted

untested, but you could add all the contour points to a single array, and call minAreaRect on it:

arr = []
for x,y,w,h in contourRects:
arr.append((x,y))
arr.append((x+w,y+h))

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners

more

I tried but it computes the average rectangles, like this (the blue rectangle) https://ibb.co/H2JjD1B

( 2018-11-26 23:42:50 -0500 )edit

you seem to do that with your "center" points, not with the actual contour ones. you also have to get rid of the 2 outlier rects at the left / right side first

( 2018-11-26 23:45:57 -0500 )edit

I tried again with actual contours ones and the results are better. Can you show me how to get rid of the outliers?

( 2018-11-26 23:51:35 -0500 )edit

maybe filter for large height ?

( 2018-11-27 00:00:12 -0500 )edit

characters have similar size, how about pick median value and filter by a multiplemedian e.g. (height > 0.75median && height < 1.25*median)

( 2018-11-27 00:25:00 -0500 )edit

@berak My application will process real time images of different heights, so I’ll have to figure out a dynamic algorithm for an optimal height. @blues I’ll try this later. Thanks!

( 2018-11-27 00:27:50 -0500 )edit

btw, we do have proper text detection methods builtin.

( 2018-11-27 00:36:40 -0500 )edit

Thanks for the information. But I found that EAST will introduce extra computational costs.

( 2018-11-27 00:41:31 -0500 )edit

@Rex Low. Can you post orginal image?

( 2018-11-29 05:41:51 -0500 )edit

@supra56 Hi, sorry for late response. I have updated the post with several test images. Please have a look at them :)

( 2018-12-02 05:59:33 -0500 )edit

Official site

GitHub

Wiki

Documentation