Ask Your Question

Detect variations of a subimage in image

asked 2015-11-15 06:45:12 -0500

jonas123 gravatar image

updated 2015-11-22 04:41:57 -0500

I'm currently trying to figure out how i can detect a subimage in an image. However i'm not sure which techniques i should apply..

Example: trying to detect finger nails in an image, such that i can get the area of those nails and calculate the color of them. In this case they are all blue, but it's possible that they are all different color.

image description image description image description

I've been suggested the two following ways:

  • HSV + SIFT/SURF + feature point cloud matching(cluster analysis)
  • for polygons instead then HSV + segmentation + polygonize + shape comparison/matching

To begin with, I don't understand how i can 'define' a finger nail. I though of photoshopping out some nails out of pictures, but since there are so many variations, it doesn't seem to be an optimal solution(different shapes/lengths/widths) and i'm not sure this is the way to go?

So i was thinking if i could somehow tag images with polygon data and location, such that it defines a nail in a picture?

In the end, i just don't know how these things are done. All help is welcome

edit retag flag offensive close merge delete


i think about an optimal solution. could you provide some more pictures?

sturkmen gravatar imagesturkmen ( 2015-11-15 15:59:22 -0500 )edit

If you simple type "blue nail polish" (or any other color) in google images, you'll get a bunch :) Thanks for helping me

jonas123 gravatar imagejonas123 ( 2015-11-16 03:14:11 -0500 )edit

@sturkmen, any ideas yet?

jonas123 gravatar imagejonas123 ( 2015-11-17 16:24:12 -0500 )edit

I've added some more pictures :) Still don't know how to do it..

jonas123 gravatar imagejonas123 ( 2015-11-22 04:42:31 -0500 )edit

You will need to apply a bag of visual words approach on fingernail objects. A tutorial can be found on GilLevi's blog.

StevenPuttemans gravatar imageStevenPuttemans ( 2015-11-23 08:12:03 -0500 )edit

Thanks for your answer, but what are exactly are fingernail objects? Just the figernails photoshopped out on a white background? I just don't know how these things are done. Could you write out some steps (e.g. which image as input/output to which algorithm, etc) which i have to follow. Thanks in advance

jonas123 gravatar imagejonas123 ( 2015-11-23 15:44:28 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2015-11-24 05:08:27 -0500

pklab gravatar image

updated 2015-11-24 05:10:57 -0500

My attempt considers that painted nails have high brightness and fingers have orientation almost similar. My code can extract images from single nails but it get some noise too. Anyway result could be used as input for a classifier.

As threshold I used the absolute peak for Brightness. I removed some noise using a threshold on Saturation in the middle of 2 biggest peaks

For example starting from this image

image description

A get histogram for HSV And get threshold for brightness and saturation (you can calculate it automatically if you have a peak detector)

image description image description

This is the binary image after some cleaning

image description

Some contour and rotated rect to get orientation and a ROI for the single detection

image description

Finally sort the detections, rotate and extract single images. From quadrant 1

image descriptionimage descriptionimage description

From quadrant 2

image descriptionimage description

here the full code. (Get FillHolesfrom @theodore answer here:

#define CL_BLU      cv::Scalar(255,  0,   0  )
#define CL_GREEN    cv::Scalar(0,    255, 0  )
#define CL_RED      cv::Scalar(0,    0,   255)
#define CL_BLACK    cv::Scalar(0,    0,   0  )
#define CL_CYAN     cv::Scalar(255,  255, 0  )
#define CL_MAGENTA  cv::Scalar(255,  0,   255)
#define CL_YELLOW   cv::Scalar(0,    255, 255)
#define CL_WHITE    cv::Scalar(255,  255, 255)

void Nails()
    cv::Scalar cl;
    cv::Mat src, dst, src_hsv, src_saturation, src_bright, bw, dist;
    src = cv::imread("../img/nails.jpg");

    vector<cv::Mat> hsv_planes;
    cv::cvtColor(src, src_hsv, cv::COLOR_BGR2HSV);
    cv::split(src_hsv, hsv_planes);
    src_saturation = hsv_planes[1];
    src_bright = hsv_planes[2];

    int thBright = 186, thSat=180;
    cv::inRange(src_hsv, cv::Scalar(0, thSat, thBright), cv::Scalar(180, 255, 255), bw);
    FillHoles(bw, bw);
    cv::bitwise_not(bw, bw);
    Morph(bw, bw, cv::MORPH_CLOSE, cv::MORPH_RECT, 3);
    Morph(bw, bw, cv::MORPH_OPEN, cv::MORPH_RECT, 1);
    cv::imshow("After TH1", bw);
    cv::imwrite("../img/bw.jpg", dst);

    // Get external contours ignoring holes
    std::vector<std::vector<cv::Point> > contours;
    cv::findContours(bw, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
    cv::RotatedRect rr;
    map <int, vector<cv::RotatedRect>> quadrant;
    vector<cv::Scalar> quadrantColor({ CL_MAGENTA, CL_YELLOW, CL_CYAN, CL_GREEN });
    double area,minArea = 0.005*bw.cols*bw.rows;
    cv::Point2f rect_points[4];
    // draw contours, select them and group by orientation
    for (int i = 0; i < contours.size(); i++)
        cv::drawContours(dst, contours, i, CL_WHITE, 1);
        //filter out small detections
        if (contours[i].size() < 10) continue;

        area = cv::contourArea(contours[i]);
        if (area < minArea) continue;

        // Get a rotated rect around the detection
        rr = cv::fitEllipse(contours[i]);

        // group detections by orientation in quadrant 
        // because there is a good chance that all fingers
        // have same orientation. 
        int q = floor(rr.angle / 90);
        // Draw rotated rect around the detection
        cl = quadrantColor[q];

        for (int j = 0; j < 4; j++)
            cv::line(dst, rect_points[j], rect_points[(j + 1) % 4], cl, 2, 8);

        cv::putText(dst, "Q" + to_string(q)+"C" + to_string(i), - cv::Point2f(3, 0), cv::FONT_HERSHEY_PLAIN, 1, CL_BLACK, 2,CV_AA);

    // Get quadrant with more detections count
    // you could consider this as related to fingers
    int cnt = 0;
    int populatedQuadrant ...
edit flag offensive delete link more



Thank you so much! I will check this out very soon :)

jonas123 gravatar imagejonas123 ( 2015-11-24 05:12:36 -0500 )edit

You could a lot around grouping quadrants and centers but my solution it's limited to pictures that contain almost nails, and you can't use it to know if a picture contains nails or not.

pklab gravatar imagepklab ( 2015-11-24 06:09:43 -0500 )edit

@pklab by the way what program is the one that you are using for the green-ish graphs, I have seen it here and in some other answers of yours and I am quite curious to learn :-p.

theodore gravatar imagetheodore ( 2015-11-24 08:25:19 -0500 )edit

@theodore Program ? Pure OpenCV code of course :) I made 3 easy funct: CreateHist, and DrawHist GetPeaks. They are unrefined to be published. For peaks detector your answer is very good option!

pklab gravatar imagepklab ( 2015-11-24 09:09:10 -0500 )edit

@pklab I see, well done ;-)

theodore gravatar imagetheodore ( 2015-11-24 16:08:29 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower


Asked: 2015-11-15 06:45:12 -0500

Seen: 1,324 times

Last updated: Nov 24 '15