Subtract operation creates result with noise

asked 2016-11-03 10:23:18 -0500

fabrizio_m gravatar image

updated 2016-11-03 10:39:14 -0500

Hi all, I'm trying to subtract two Mat instances to obtain a result that represents a person's signature. Unfortunately, my resulting Mat seems to show noise along the top edge. Here are the source images and output:

The templateToMatch: C:\fakepath\template.png

The original source from which the ROI will be extracted, referred to as "formImage" in the code:C:\fakepath\sep-ira-completed-handwritten-p5.png

The ROI extracted via matchTemplate and Rect:C:\fakepath\signed.png

The result of running subtract on the template and extracted ROI: C:\fakepath\difference.png

As you can see, difference.png shows noise creeping down over the actual signature.

Can someone have a look at my implementation and tell me what I may be doing wrong? Is there a way I can mitigate this, if not avoid it altogether?

Here's where the subtraction takes place:

cv::Mat extractSignatureFromRegion(cv::Mat & templateToMatch, cv::Mat & formImage, bool* sigFound) {
    Mat signature;
    Mat result;
    bool hasSignature = false;
    Mat sigRegion = getSignatureRegion(templateToMatch, formImage);

    subtract(templateToMatch, sigRegion, signature);
    cvtColor(signature, signature, CV_BGR2GRAY);
    if (countNonZero(signature) > 0) {
        hasSignature = true;
    }
    *sigFound = hasSignature;
    return signature;
}

Here's the getSignatureRegion that uses template matching to extract the potentially signed region in the source doc:

    cv::Mat getSignatureRegion(cv::Mat & templateToMatch, cv::Mat & formImage) {
    //Create result matrix from template and original source
    Mat resultImg;
    Mat resultMatrix;
    Mat regionMatchingTemplate;
    Mat result;
    double minVal;
    double maxVal;
    Point minLoc;
    Point maxLoc;
    Point matchLoc;

    int resultColumns = formImage.cols - templateToMatch.cols + 1;
    int resultRows = formImage.rows - templateToMatch.rows + 1;
    resultMatrix.create(resultColumns, resultRows, CV_32FC1);


    // Match template and normalize
    matchTemplate(formImage, templateToMatch, resultMatrix, CV_TM_SQDIFF);
    normalize(resultMatrix, resultMatrix, 0, 1, NORM_MINMAX, -1, Mat());
    minMaxLoc(resultMatrix, &minVal, &maxVal, &minLoc, &maxLoc, Mat());
    matchLoc = minLoc;
    regionMatchingTemplate = formImage(Rect(matchLoc.x, matchLoc.y, templateToMatch.cols, templateToMatch.rows));
    regionMatchingTemplate.copyTo(result);
    return result;
}

Here's another example where this time, I used a template to extract a region from an empty form where no signature is present in the ROI. I would expect the subtract operation to result in a nearly empty Mat, but again there is "noise" along the bottom edge of the result:

Unsigned form: C:\fakepath\rollover-signature-page-unsigned.jpg

Template to find ROI of empty signature box: C:\fakepath\template.png

ROI that was extracted from unsigned form: C:\fakepath\signed.png

And finally, the subtract result with a weird black rectangled along bottom edge: C:\fakepath\difference.png

edit retag flag offensive close merge delete

Comments

I haven't ever seen anything like this except possibly some only partially saved pictures. Are you exiting before it finishes saving? I'll take a look at this later today and try to replicate.

Tetragramm gravatar imageTetragramm ( 2016-11-03 11:01:45 -0500 )edit
1

Cannot replicate. It works as expected, and I didn't alter your code at all.

Tetragramm gravatar imageTetragramm ( 2016-11-03 18:20:34 -0500 )edit

Thanks a lot for testing this out. The input data is just getting passed into the code that calls opencv from a view controller in an iOS app that loads the input images from the device's filesystem. And then I return the results (result, regionmatchingTemplate Mat instances) back up for display to the user. Just curious, which platform did you test this on? I may switch to a different reference platform, as I've definitely had issues on iOS.

fabrizio_m gravatar imagefabrizio_m ( 2016-11-04 06:52:01 -0500 )edit

I tested in C++ on Windows. Should work fine on anything I would think. Whatever the problem is, I don't think it's OpenCV. There's some event or process going on either loading or saving the data that isn't finishing before you start doing things.

Tetragramm gravatar imageTetragramm ( 2016-11-04 07:46:35 -0500 )edit