Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.`cv::Mat img; UIImageToMat(input, img, false); cv::Mat grayed; cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

`Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

click to hide/show revision 2
retagged

updated 2015-06-04 00:42:22 -0500

berak gravatar image

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.`cv::Mat img; UIImageToMat(input, img, false); cv::Mat grayed; cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

`Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.`cv::Mat img; UIImageToMat(input, img, false); cv::Mat grayed; cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

`Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.`cv::Mat result.

cv::Mat img;
 UIImageToMat(input, img, false);
 cv::Mat grayed;
 cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

`Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

click to hide/show revision 5
retagged

updated 2015-06-04 05:46:36 -0500

berak gravatar image

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.

cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

click to hide/show revision 6
retagged

updated 2015-06-04 05:47:16 -0500

berak gravatar image

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.

cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.

cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

Step 1[Step 1] http://s6.postimg.org/sflk5g9sx/step1.png

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2[Step 2]http://s6.postimg.org/tj5oheug1/Step2.png

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3[Step 3]http://s6.postimg.org/yvuivjich/step3.png

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

EDIT: UPDATED CODE

    cv::Mat img, output;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));
cv::Mat temp;
cv::morphologyEx(grayed, temp, MORPH_RECT, element);

cv::GaussianBlur(temp, temp, cv::Size(7, 7), 2);
cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 207, -2);

cv::medianBlur(temp, temp, 3);
cv::Mat binary;
double otsu_thresh_val = cv::threshold(temp, binary, 225, 255, cv::THRESH_BINARY_INV);
cv::erode(binary, binary, element);
cv::dilate(binary, binary, element);

if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

cv::Mat canny_output;
Canny( binary, canny_output, lower_thresh_val, high_thresh_val, 3 );

bitwise_not(canny_output, canny_output);
const int connectivity_8 = 4;
Mat labels, stats, centroids;
int nLabels = connectedComponentsWithStats(canny_output, labels, stats, centroids, connectivity_8, CV_32S);
NSLog(@"nLabels %i", nLabels);
return MatToUIImage(canny_output);

Step 4