Extract text and bounding-box using OpenCV

asked 2018-04-26 03:11:01 -0500

Kailash gravatar image

I am trying to find the bounding boxes (rectangles) of text in an image. So, I like to scan entire image at once and get the all textual area. Issue is few texts are ignored and few are goruped together. The grouping should be word wise. It should not be grouped together having more than one word. Each word should have its own rectangle. I am using code given below.

    int COCR::method_gradient(int nonGradient, int showOutput)

Mat large = imread(INPUT_FILE);
Mat rgb;

rgb = large;  //--> change

Mat small;
cvtColor(rgb, small, CV_BGR2GRAY);

// morphological gradient
Mat grad;
Mat morphKernel;

if (!nonGradient)
    morphKernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
    morphologyEx(small, grad, MORPH_GRADIENT, morphKernel);
    grad = small;

// binarize
Mat bw;
threshold(grad, bw, 0.0, 255.0, THRESH_BINARY | THRESH_OTSU);

// connect horizontally oriented regions
Mat connected;
morphKernel = getStructuringElement(MORPH_RECT, Size(9, 1));
morphologyEx(bw, connected, MORPH_CLOSE, morphKernel);

// find contours
Mat mask = Mat::zeros(bw.size(), CV_8UC1);
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(connected, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));

if (nonGradient)
    rgb = MatGradient;

// filter contours
for (int idx = 0; idx >= 0; idx = hierarchy[idx][0])
    Rect rect = boundingRect(contours[idx]);
    Mat maskROI(mask, rect);
    maskROI = Scalar(0, 0, 0);

    // fill the contour
    drawContours(mask, contours, idx, Scalar(255, 255, 255), CV_FILLED);

    // ratio of non-zero pixels in the filled region
    double r = (double)countNonZero(maskROI) / (rect.width*rect.height);

    if (r > .25 /* assume at least 25% of the area is filled if it contains text */
        (rect.height > 8 && rect.width > 8) 
        rectangle(rgb, rect, Scalar(0, 255, 0), 2);

if (!nonGradient)
    MatGradient = rgb;

if (showOutput)
    imwrite(OUTPUT_FOLDER_PATH, rgb);

return 0;

// calling
COCR obj_ocr;   
obj_ocr.method_gradient(0, 0);
obj_ocr.method_gradient(1, 1);


Please notice the red part. output_img1


Please see red part. output_img2

please advise how can I rectify the missing area shown/marked in Red eclipse.

edit retag flag offensive close merge delete


please be so kind, and remove your duplicate questions, thank you.

berak gravatar imageberak ( 2018-04-26 03:17:01 -0500 )edit

btw its actually normal that it does not work for the not identified regions. Your contours and morphological operations all depend on the fact that you have black text againt a brighter background and there it is inversed. I am convinced that if you do an invertion of your input image, the contacts will be easily detected :)

StevenPuttemans gravatar imageStevenPuttemans ( 2018-04-27 07:00:20 -0500 )edit