# How to merge nearby rectangles

I am working on character segmentation and looking to merge nearby characters into a box. I have successfully found the contours but I am not sure how to find a straight line between the top and bottom points.

def findContours(self, image):
contour_img = image.copy()
vis = image.copy()
vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
_, contours, hierarchy = cv2.findContours(contour_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

contoursBBS = []
for contour in contours:
[x, y, w, h] = cv2.boundingRect(contour)

if w > 100 or h < 8:
continue

contoursBBS.append([x, y, w, h, w*h])
contoursBBS = np.array(contoursBBS)

# reject outliers based on width, height, and area
rejectH = self.rejectOutliers(contoursBBS, 3, m=4)
rejectW = self.rejectOutliers(rejectH, 2, m=4)
rejectA = self.rejectOutliers(rejectW, 4, m=4)

contourRects = []
for c in rejectA:
[x, y, w, h, a] = c
if w < 9 or h < 15 or a < 300 or a > 6000:
continue
contourRects.append(c)

for i, rect in enumerate(contourRects):
[x, y, w, h, a] = rect
topCenter, bottomCenter = (int((2*x + w)/2), int(y)), (int((2*x + w)/2), int(y+h))

print("X: {:4d}  Y:  {:4d}  W: {:4d}  H: {:4d}  A: {:4d}".format(x, y, w, h, a))

cv2.rectangle(vis, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.circle(vis, topCenter, 2, (0, 0, 255), 2)
cv2.circle(vis, bottomCenter, 2, (0, 0, 255), 2)


Result What I am hoping to achieve Some test images (requested by @supra56)

edit retag close merge delete

Sort by » oldest newest most voted

untested, but you could add all the contour points to a single array, and call minAreaRect on it:

arr = []
for x,y,w,h in contourRects:
arr.append((x,y))
arr.append((x+w,y+h))

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners

more

I tried but it computes the average rectangles, like this (the blue rectangle) https://ibb.co/H2JjD1B

you seem to do that with your "center" points, not with the actual contour ones. you also have to get rid of the 2 outlier rects at the left / right side first

I tried again with actual contours ones and the results are better. Can you show me how to get rid of the outliers?

characters have similar size, how about pick median value and filter by a multiplemedian e.g. (height > 0.75median && height < 1.25*median)

@berak My application will process real time images of different heights, so I’ll have to figure out a dynamic algorithm for an optimal height. @blues I’ll try this later. Thanks!

Thanks for the information. But I found that EAST will introduce extra computational costs.

1

@supra56 Hi, sorry for late response. I have updated the post with several test images. Please have a look at them :)

Append all your rectangles in rectList and use cv2.boundingRect() to get the bigger rectangle that enclose all sub rectangles. Let me know if it works for you. Thanks berak's for the piece of code


arr = []
for x,y,w,h in rectList:
arr.append((x,y))
arr.append((x+w,y+h))

x,y,w,h = cv2.boundingRect(np.asarray(arr))
cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),1)

more

I found better solution without iterating through contours - by using numpy concatenate() function:

concat = np.concatenate(contours)
hulls = cv2.convexHull(concat)


Resulting hulls contour can be further approximated by cv2.approxPolyDP() function in order to get bounding rectangle.

more

Official site

GitHub

Wiki

Documentation