OpenCV 4.0 CNN for text detection

asked 2018-10-28 14:27:28 -0500

jbtr142 gravatar image

updated 2018-10-28 16:21:32 -0500

I'm attempting to use OpenCV for text detection of Canadian apartment floor plans for the purpose of building text boxes which can be run through an OCR. The current code works quite well for some but less well for other images

img = cv2.imread("Image.jpg")
mask = np.zeros(img.shape, dtype=np.uint8)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, threshold = cv2.threshold(gray,150,255,cv2.THRESH_BINARY_INV)
_, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)

ROI = []

for cnt in contours:
    x,y,w,h = cv2.boundingRect(cnt)
    if h < 20:
         cv2.drawContours(mask, [cnt], 0, (255,255,255), 1)
kernel = np.ones((7,7),np.uint8)
dilation = cv2.dilate(mask,kernel,iterations = 1)
gray_d = cv2.cvtColor(dilation, cv2.COLOR_BGR2GRAY)
_, threshold_d = cv2.threshold(gray_d,150,255,cv2.THRESH_BINARY)
_, contours_d, hierarchy = cv2.findContours(threshold_d,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)     

for cnt in contours_d:
    x,y,w,h = cv2.boundingRect(cnt)
    if w > 35:
         roi_c = img[y:y+h, x:x+w]

config = ("-l eng --oem 3 --psm 6")
for R in ROI:
    text = pytesseract.image_to_string(R, config=config)

The output for two images is:



The main questions are: 1) Is there anything in OpenCV 4 that may be of use to improving the accuracy of the text detector? For example, there is TextDetectorCNN, but I have minimal experience in it and there is no example of its implmentation in Python. 2) Is there any other OpenCV techniques that may be useful? Is there a way to combine boxes that are close together. For example in the second image, combining the room name and its dimensions into one box.

Raw images attached:

C:\fakepath\11054.jpg C:\fakepath\30005.jpg C:\fakepath\30010.jpg C:\fakepath\44749.jpg

edit retag flag offensive close merge delete



Can you please provide a few examples of input images, eh?

sjhalayka gravatar imagesjhalayka ( 2018-10-28 15:45:09 -0500 )edit

I've attached a few raw samples in the edit.

jbtr142 gravatar imagejbtr142 ( 2018-10-28 16:21:52 -0500 )edit

Thank you!

sjhalayka gravatar imagesjhalayka ( 2018-10-28 16:24:19 -0500 )edit

you could try with this or maybe even with this

berak gravatar imageberak ( 2018-10-29 02:43:54 -0500 )edit