Ask Your Question
1

How to merge nearby rectangles

asked 2018-11-26 23:21:19 -0500

Rex Low gravatar image

updated 2018-12-02 05:58:55 -0500

I am working on character segmentation and looking to merge nearby characters into a box. I have successfully found the contours but I am not sure how to find a straight line between the top and bottom points.

def findContours(self, image):
    contour_img = image.copy()
    vis = image.copy()
    vis = cv2.cvtColor(vis, cv2.COLOR_GRAY2BGR)
    _, contours, hierarchy = cv2.findContours(contour_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    contoursBBS = []
    for contour in contours:
        [x, y, w, h] = cv2.boundingRect(contour)

        if w > 100 or h < 8:
            continue

        contoursBBS.append([x, y, w, h, w*h])
    contoursBBS = np.array(contoursBBS)

    # reject outliers based on width, height, and area
    rejectH = self.rejectOutliers(contoursBBS, 3, m=4)
    rejectW = self.rejectOutliers(rejectH, 2, m=4)
    rejectA = self.rejectOutliers(rejectW, 4, m=4)

    contourRects = []
    for c in rejectA:
        [x, y, w, h, a] = c
        if w < 9 or h < 15 or a < 300 or a > 6000:
            continue
        contourRects.append(c)

    for i, rect in enumerate(contourRects):
        [x, y, w, h, a] = rect
        topCenter, bottomCenter = (int((2*x + w)/2), int(y)), (int((2*x + w)/2), int(y+h))

        print("X: {:4d}  Y:  {:4d}  W: {:4d}  H: {:4d}  A: {:4d}".format(x, y, w, h, a))


        cv2.rectangle(vis, (x, y), (x+w, y+h), (0, 255, 0), 2)
        cv2.circle(vis, topCenter, 2, (0, 0, 255), 2)
        cv2.circle(vis, bottomCenter, 2, (0, 0, 255), 2)

Result

image description

What I am hoping to achieve

image description

Some test images (requested by @supra56)

  1. https://ibb.co/bgtyypp
  2. https://ibb.co/XtdHgWg
  3. https://ibb.co/5R4QVTR
  4. https://ibb.co/gD8KymR (very challenging)
  5. https://ibb.co/k5YDT1Q
edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted
1

answered 2018-11-26 23:33:55 -0500

berak gravatar image

untested, but you could add all the contour points to a single array, and call minAreaRect on it:

arr = []
for x,y,w,h in contourRects:
      arr.append((x,y))
      arr.append((x+w,y+h))

box = cv.minAreaRect(np.asarray(arr))
pts = cv.boxPoints(box) # 4 outer corners
edit flag offensive delete link more

Comments

I tried but it computes the average rectangles, like this (the blue rectangle) https://ibb.co/H2JjD1B

Rex Low gravatar imageRex Low ( 2018-11-26 23:42:50 -0500 )edit

you seem to do that with your "center" points, not with the actual contour ones. you also have to get rid of the 2 outlier rects at the left / right side first

berak gravatar imageberak ( 2018-11-26 23:45:57 -0500 )edit

I tried again with actual contours ones and the results are better. Can you show me how to get rid of the outliers?

Rex Low gravatar imageRex Low ( 2018-11-26 23:51:35 -0500 )edit

maybe filter for large height ?

berak gravatar imageberak ( 2018-11-27 00:00:12 -0500 )edit

characters have similar size, how about pick median value and filter by a multiplemedian e.g. (height > 0.75median && height < 1.25*median)

blues gravatar imageblues ( 2018-11-27 00:25:00 -0500 )edit

@berak My application will process real time images of different heights, so I’ll have to figure out a dynamic algorithm for an optimal height. @blues I’ll try this later. Thanks!

Rex Low gravatar imageRex Low ( 2018-11-27 00:27:50 -0500 )edit

btw, we do have proper text detection methods builtin.

berak gravatar imageberak ( 2018-11-27 00:36:40 -0500 )edit

Thanks for the information. But I found that EAST will introduce extra computational costs.

Rex Low gravatar imageRex Low ( 2018-11-27 00:41:31 -0500 )edit

@Rex Low. Can you post orginal image?

supra56 gravatar imagesupra56 ( 2018-11-29 05:41:51 -0500 )edit

@supra56 Hi, sorry for late response. I have updated the post with several test images. Please have a look at them :)

Rex Low gravatar imageRex Low ( 2018-12-02 05:59:33 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-11-26 23:21:19 -0500

Seen: 56 times

Last updated: Dec 02