Ask Your Question

mapping 2 document [closed]

asked 2016-07-03 11:56:43 -0600

vincentdo2310 gravatar image

image description(http://)(/upfiles/1467564860315774.jpg)(/upfiles/14675648549169814.jpg)Hi guys, I'm fairly new to opencv. Currently I'm having a project to mapping 2 form of document ( the template and the written one ) to detect the handwriting. I scanned 2 document using perspective transform and get the result like below. Question is : how do i detect those differences, im just asking for the approach, i would appreciate for any helps

edit retag flag offensive reopen merge delete

Closed for the following reason duplicate question by vincentdo2310
close date 2016-07-11 17:10:35.538222


Not quite clear what do you want to do. Do you mean you want to teach the computer know how to read the words(OCR)?

stereomatching gravatar imagestereomatching ( 2016-07-03 13:50:28 -0600 )edit

No, i just want to detect where the handwriting is, not to read it

vincentdo2310 gravatar imagevincentdo2310 ( 2016-07-03 22:51:14 -0600 )edit

I think that the solution below is more to locate text, why you want only the written parts. What I would do is, apply keypoint detection on template, then on the filled in one, do a registration between points and register the filled in one on the template. Then do a subtraction of both and the text should remain.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-07-05 04:22:56 -0600 )edit

it is what i think, but there is small shifting modification after i scanned the document => when i subtract the overlay zone will still remain alot of noise

vincentdo2310 gravatar imagevincentdo2310 ( 2016-07-05 08:58:25 -0600 )edit

the registration part applies a non rigid affine transform to register both documents, so that shouldn't be a problem...

StevenPuttemans gravatar imageStevenPuttemans ( 2016-07-05 09:03:36 -0600 )edit

ok thank you, i will work on it again

vincentdo2310 gravatar imagevincentdo2310 ( 2016-07-05 09:37:11 -0600 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2016-07-04 03:25:01 -0600

stereomatching gravatar image

updated 2016-07-04 03:54:09 -0600

I would use binary threshold, morphology and contours to solve this problem.

Step 1 : get the copy with signature

cv::Mat const map_01 = cv::imread("map01.jpg");

image description

Step 2 : blur the image and convert it to binary image

cv::Mat binarize;
cv::cvtColor(map_01, binarize, CV_BGR2GRAY);
cv::GaussianBlur(map_01, map_01, {5,5}, 3,3);
cv::threshold(binarize, binarize, 0, 255,

Step 3 : Use morphology to remove small blobs and make text blobs more obvious

auto mrect = cv::getStructuringElement(cv::MORPH_RECT, {3,3});
cv::erode(binarize, binarize, mrect);
mrect = cv::getStructuringElement(cv::MORPH_RECT, {7,5});
cv::morphologyEx(binarize, binarize, cv::MORPH_DILATE, mrect, {-1,-1}, 7);

Step 4 : find contours and remove those with invalid aspect ratio

std::vector<std::vector<cv::Point>> contours;
cv::findContours(binarize.clone(), contours, CV_RETR_EXTERNAL,
auto rit = std::remove_if(std::begin(contours), std::end(contours),
               [](auto const &ct)
    auto const rect = cv::boundingRect(ct);
    return rect.height > rect.width;
contours.erase(rit, std::end(contours));

Step 5 : Find the data of each rectangle

for(auto const &ct : contours){
    //get the information of each rect
    cv::Mat temp = map_01.clone();
    cv::rectangle(temp, cv::boundingRect(ct), {255,0,0});
    cv::imshow("temp", temp);

The bounding rects found by this solution are

image description

Step 6 : Extract text location according to the location and size of the bounding rects, this step is quite tedious.

Not a very good one, I believe there exist better solution, like find the difference between two images.

Source codes locate at github.

Edit : If the format and size of the are always the same, you can predefined the bounding rect of text candidates, extract the roi from predefined bounding rect, detect the binary value of those roi to find out the regions filled or not.

edit flag offensive delete link more


thankyou so much, i will take a closer look to your code right now I have tested the method like find the difference but there is small shifting modification after i scanned the document

vincentdo2310 gravatar imagevincentdo2310 ( 2016-07-04 06:46:36 -0600 )edit

Question Tools

1 follower


Asked: 2016-07-03 11:56:43 -0600

Seen: 364 times

Last updated: Jul 04 '16