Extracting stick-it notes from image problem

asked 2018-01-05 04:44:27 -0500

piowes86 gravatar image

Hi All!

from 2-3 weeks I am trying to extract stick-it notes from image. Until now I am able to extract some of them with better or worse results, but for some images it doesn't work at all. When there is one colour background and stick-it notes are different colour, my program is able to detect all of the notes. When the background is different in lots of part of background the problems begin.

To detect stick-it notes I decided to use findContours method together with cvtColor + blur + Canny/Threshold methods.

My code for detecting notes looks like below:

std::vector<cv::mat> OpenCVWrapper::getImagesOfObjects(cv::Mat img) {

std::vector<cv::Mat> images;

cv::Mat src_img = img;
cv::Mat dst;

int thresh = 100;
cv::RNG rng(12345);

std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;

cv::cvtColor(src_img, dst, cv::COLOR_RGB2GRAY);
cv::Canny(src_img, dst, thresh, thresh*2);
src_img = dst;

cv::Mat kernel = cv::getStructuringElement(cv::MORPH_RECT, cv::Point(7, 7));
cv::morphologyEx(src_img, dst, cv::MORPH_CLOSE, kernel);

cv::findContours(dst, contours, hierarchy, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);

cv::Rect box;
cv::Mat image;

int wholeImagePixelsNumber = int(img.size().width * img.size().height);

for(int i=0; i< contours.size(); i++)
{
    double percentOfWholeArea = (contourArea(contours[i])*100)/wholeImagePixelsNumber;

    if ( percentOfWholeArea > 0.5 ) {
        box = boundingRect(contours[i]);
        image = img(cv::Rect(box.x, box.y, box.width, box.height)).clone();
        images.push_back(image);
    }
}

return images;

}

I am struggling now with that how to get better results when the background is noisy or the contrast between notes and background is not easy to detect.

I will appreciate any help.

Some examples:

Image 1 - program was able to detect all of the stick-it notes

image description

image description

image description

Image 2 - program has detected 10 elements but two of them like second example

image description

image description

image description

Image 3 - program was not able to detect anything

image description

I thing that the key is to make image much contrast by focusing only on stick-it notes. But the question is how to do that? I was trying to use HSV transformation but later I was not able to use performed image with findContours method.

I stucked. If anyone knows or have idea how to solve the problem please let me know.

Thank you!

edit retag flag offensive close merge delete

Comments

If you know the range of sticky note colors beforehand, than a simply pre-filter allows you to segment them from the others, giving you a clear mask. For now you are not doing anything with the fact that notes have a flashy color!

StevenPuttemans gravatar imageStevenPuttemans ( 2018-01-08 04:00:11 -0500 )edit