How can I use OpenCV to detect all the text in an image, I want to to be able to detect "blocks" of texts individually. Then pass the the recognized blocks into tesseract.
Here is an example, if I were to scan this I would want to scan the paragraphs separately, not go from left to right which is what tesseract does.
Well this a little complex problem and is an active area for research. For the fact I am working on the same problem. Here is the procedure I have adopted
Now that you have your image loaded
2 a) Extract pixels representing individual letters of text
Approach 1: SWT (Stroke Width Transform) REF SWT needs to know if your text is Light-on-dark background or Dark-on-light background, if you do not know this then you have to make two passes of this algorithm.
Approach 2: Flood fill your image repetitively until all pixel are covered. For every seed pixel mark the pixel that got included in the floodfill and do not run floodfill for these pixel again.
For each seed pixel collected the relevant pixel and call the collection a component. Filter out non-relevant components using any appproach similar to the case of SWT.
Group the components to make text words, then to text lines and finally into text blocks.
Pass the text block to tesseract.
Current implementation of floodfill in opencv check here returns only the number of pixel marked for a seed pixel and not the pixel location. Hence to get pixel location in order form components some modification needs to be made. The marked pixel location can be collected in a Boost::unordered_map for better performance.
Also this is a github project that extracts text components from images using SWT approach.
to solve this kind of problems i want to share an approach based on cv::erode
with the code below i tried to find text blocks. the result is OK by manuel parameters of cv::erode
cv::Mat kernel = cv::Mat::ones(10, 5, CV_8U);
erode(src_gray,src_gray, kernel, Point(-1,-1),2);
with a bit effort it is possible to develop an algorithm for manually entered values.
image after binarization and cv::erode
final result
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace cv;
using namespace std;
int main( int, char** argv )
{
Mat src,src_gray;
src = imread("27411.jpg");
if (src.empty())
{
cerr << "No image supplied ..." << endl;
return -1;
}
cvtColor( src, src_gray, COLOR_BGR2GRAY );
src_gray = src_gray >127;
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
cv::Mat kernel = cv::Mat::ones(10, 5, CV_8U);
erode(src_gray,src_gray, kernel, Point(-1,-1),2);
imshow( "src_gray", src_gray );
findContours( src_gray, contours, hierarchy, RETR_LIST, CHAIN_APPROX_SIMPLE, Point(0, 0) );
for( size_t i = 0; i< contours.size(); i++ )
{
Scalar color = Scalar( 0,255,0 );
Rect R = boundingRect(Mat(contours[i]));
rectangle(src,R,color);
}
imshow( "result", src );
waitKey(0);
return(0);
}
Asked: 2014-01-29 15:13:21 -0600
Seen: 19,674 times
Last updated: Aug 07 '15