Revision history [back]

Gap filling in letters

Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading. This is the original image without cropping

By performing the following on the image (cropped & orientation fixed), we get the following result.`cv::Mat img; UIImageToMat(input, img, false); cv::Mat grayed; cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

` Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.

Regards.

Gap filling in letters

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

` Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Regards.

Gap filling in letters

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

` Step 1

Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:

cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration

Step 2

As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:

    Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}

Step 3

Regards.

Gap filling in letters

By performing the following on the image (cropped & orientation fixed), we get the following ~~result.`cv::Mat~~ result.

cv::Mat img;
 UIImageToMat(input, img, false);
 cv::Mat grayed;
 cv::cvtColor(img, grayed, COLOR_BGR2GRAY);
 COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
 `
 Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:
 cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration
 
 As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:
     Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}
 
 Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.
 Regards.

          5    retagged 
      updated 
2015-06-04 05:46:36 -0600
 
    berak 
 32993  ●7 ●81 ●312  
 
 
 
  
 
 
 
  Gap filling in letters
 Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.
 By performing the following on the image (cropped & orientation fixed), we get the following result. 
 cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
 
 Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:
 cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration
 
 As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:
     Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}
 
 Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.
 Regards.
 
 
 
          6    retagged 
      updated 
2015-06-04 05:47:16 -0600
 
    berak 
 32993  ●7 ●81 ●312  
 
 
 
  
 
 
 
  Gap filling in letters
 Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.
 By performing the following on the image (cropped & orientation fixed), we get the following result. 
 cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
 
 Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:
 cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration
 
 As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:
     Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}
 
 Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.
 Regards.
 
 
 
          7    No.7 Revision 
      updated 
2015-06-04 09:11:24 -0600
 
 
 
  
 
 
 
  Gap filling in letters
 Reading text from image is a part of iOS application, I'm currently in. We are using OpenCV to process image before sending it to tesseract. I have read in an article, it is advisable to have black text on white background to achieve higher success rate in reading.
 By performing the following on the image (cropped & orientation fixed), we get the following result. 
 cv::Mat img;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));

cv::Mat temp;
cv::morphologyEx(grayed, temp, BORDER_WRAP, element);

cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 15, -2);
double otsu_thresh_val = threshold(temp, temp, 0, 255, THRESH_BINARY_INV | THRESH_OTSU);
if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;
 [Step 1] http://s6.postimg.org/sflk5g9sx/step1.png
 Now, the remaining part is filling the letters with black color. Read dilation after canny makes the border line thicker , which I thought would fill the letters inwards. But I'm proved wrong. Code:
 cv::Mat canny_output;
Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
cv::dilate( canny_output, canny_output, element, cv::Point(-1, -1), 1);//1 iteration
 [Step 2]http://s6.postimg.org/tj5oheug1/Step2.png
 As another approach, I have tried to find contours and fill. Applied canny w/o dilation. Code:
     Canny( temp, canny_output, lower_thresh_val, high_thresh_val, 3 );
vector<vector<cv::Point> > contours;
vector<Vec4i> hierarchy;
RNG rng(12345);
findContours( canny_output, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE, cv::Point(1, 1));
cv::Mat drawing = Mat::zeros( canny_output.size(), CV_8UC3 );

vector<cv::Point> approxShape;
for(int i = 0; i < contours.size(); i++){
    approxPolyDP(contours[i], approxShape, arcLength(Mat(contours[i]), true)*0.04, true);
    drawContours(drawing, contours, i, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)), FILLED);   // fill random color
}
 [Step 3]http://s6.postimg.org/yvuivjich/step3.png
 Which fills the inner holes. This is my first try at OpenCV, probably i'm misunderstanding the codes and its intended concept. It would be of great help, if someone could point me out the mistakes and/or in which direction i should proceed further.
 Regards.
  EDIT: UPDATED CODE
     cv::Mat img, output;
UIImageToMat(input, img, false);
cv::Mat grayed;
cv::cvtColor(img, grayed, COLOR_BGR2GRAY);

int erosion_size = 12.0;
cv::Mat element = cv::getStructuringElement(0, cv::Size(erosion_size + 1, erosion_size + 1), cv::Point(1, 1));
cv::Mat temp;
cv::morphologyEx(grayed, temp, MORPH_RECT, element);

cv::GaussianBlur(temp, temp, cv::Size(7, 7), 2);
cv::bitwise_not(temp, temp);
adaptiveThreshold(~temp, temp, 255, ADAPTIVE_THRESH_MEAN_C, 0, 207, -2);

cv::medianBlur(temp, temp, 3);
cv::Mat binary;
double otsu_thresh_val = cv::threshold(temp, binary, 225, 255, cv::THRESH_BINARY_INV);
cv::erode(binary, binary, element);
cv::dilate(binary, binary, element);

if (otsu_thresh_val == 0)
    otsu_thresh_val = 100;
double high_thresh_val  = otsu_thresh_val, lower_thresh_val = otsu_thresh_val * 0.5;

cv::Mat canny_output;
Canny( binary, canny_output, lower_thresh_val, high_thresh_val, 3 );

bitwise_not(canny_output, canny_output);
const int connectivity_8 = 4;
Mat labels, stats, centroids;
int nLabels = connectedComponentsWithStats(canny_output, labels, stats, centroids, connectivity_8, CV_32S);
NSLog(@"nLabels %i", nLabels);
return MatToUIImage(canny_output);