Gap filling in letters
Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.
By performing the following on the image (cropped & orientation fixed), we get the following result.
cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);
int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));
cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);
cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
otsu_thresh_val = 100;
double high_thresh_val = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
[Step 1] http://s6.postimg.org/sflk5g9sx/step1...
Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:
cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration
[Step 2]http://s6.postimg.org/tj5oheug1/Step2.png
As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );
vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED); // fill random color
}
[Step 3]http://s6.postimg.org/yvuivjich/step3.png
Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.
Regards.
EDIT: UPDATED CODE
cv::Mat img, output;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);
int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));
cv::Mat temp;
cv::morphologyEx(grayed, temp, MORPH_RECT, element);
cv::GaussianBlur(temp, temp, cv::Size(7, 7), 2);
cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 207, -2);
cv::medianBlur(temp, temp, 3);
cv::Mat binary;
double otsu_thresh_val = cv::threshold(temp, binary, 225, 255, cv::THRESH_BINARY_INV);
cv::erode(binary, binary, element);
cv::dilate(binary, binary, element);
if (otsu_thresh_val == 0)
otsu_thresh_val = 100;
double high_thresh_val = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
cv::Mat canny_output;
Canny( binary, canny_output, lower_thresh_val, high_thresh_val, 3 );
bitwise_not(canny_output, canny_output);
const int connectivity_8 = 4;
Mat labels, stats, centroids;
int nLabels = connectedComponentsWithStats(canny_output, labels, stats ...
@LBerger , wow, a pipeline ;) is there more to look at ?
@LBerger Thank you! The result image is too perfect. But I couldn't understand the xml, as I'm fairly new to opencv. Is it possible for a pseudo-code or something?
@berak In my program I save all sequence operation in xml file (some problems with yml). After loading this xml file I can execute same sequence for an another image or for a video.