Ask Your Question
3

Gap filling in letters

asked 2015-06-04 00:35:38 -0500

Nina gravatar image

updated 2015-06-04 09:11:24 -0500

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.

cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

[Step 1] http://s6.postimg.org/sflk5g9sx/step1...

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

[Step 2]http://s6.postimg.org/tj5oheug1/Step2.png

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

[Step 3]http://s6.postimg.org/yvuivjich/step3.png

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

EDIT: UPDATED CODE

    cv::Mat img, output;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));
cv::Mat temp;
cv::morphologyEx(grayed, temp, MORPH_RECT, element);

cv::GaussianBlur(temp, temp, cv::Size(7, 7), 2);
cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 207, -2);

cv::medianBlur(temp, temp, 3);
cv::Mat binary;
double otsu_thresh_val = cv::threshold(temp, binary, 225, 255, cv::THRESH_BINARY_INV);
cv::erode(binary, binary, element);
cv::dilate(binary, binary, element);

if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

cv::Mat canny_output;
Canny( binary, canny_output, lower_thresh_val, high_thresh_val, 3 );

bitwise_not(canny_output, canny_output);
const int connectivity_8 = 4;
Mat labels, stats, centroids;
int nLabels = connectedComponentsWithStats(canny_output, labels, stats ...
(more)
edit retag flag offensive close merge delete

Comments

@LBerger , wow, a pipeline ;) is there more to look at ?

berak gravatar imageberak ( 2015-06-04 07:35:50 -0500 )edit

@LBerger Thank you! The result image is too perfect. But I couldn't understand the xml, as I'm fairly new to opencv. Is it possible for a pseudo-code or something?

Nina gravatar imageNina ( 2015-06-04 07:40:08 -0500 )edit

@berak In my program I save all sequence operation in xml file (some problems with yml). After loading this xml file I can execute same sequence for an another image or for a video.

LBerger gravatar imageLBerger ( 2015-06-04 07:54:36 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
3

answered 2015-06-04 07:29:43 -0500

LBerger gravatar image

updated 2015-06-04 07:50:07 -0500

Here my results with these operations in xml format (cvtColor, gaussianBlur,bitwise-not,adaptativethreshold)

1 cvtcolor with Code=0

2 gaussianblur with size(7,7) and sigma =2 (I don't think it is necessary)

3 bitwise-not

4 adaptativeThreshold with blockSize=207 and adaptativeMethod=threshold mean of neighbourhood

edit flag offensive delete link more

Comments

For cvtColor code 0 is COLOR_BGR2BGRA? It works only if I change it to Grayscale. And the text "2" on top left and its background is just reversed. Is there any solution for it?

Nina gravatar imageNina ( 2015-06-04 08:09:30 -0500 )edit

you can use HSV system and works with H. But I don't think all yours letters are green or white. You can try with connected components and use this post answer

LBerger gravatar imageLBerger ( 2015-06-04 08:17:50 -0500 )edit

Thank you! I will try the suggested solution..

Nina gravatar imageNina ( 2015-06-04 08:22:09 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2015-06-04 00:35:38 -0500

Seen: 1,062 times

Last updated: Jun 04 '15