Ask Your Question
1

Detection of stones (rocks) on field surface

asked 2016-04-06 15:26:16 -0500

egdaves gravatar image

updated 2016-04-14 02:53:38 -0500

pklab gravatar image

A project I have been thinking about for some time is a robotic stone picker to clear my fields of stones (rocks). These vary in size from 75 mm to 350 mm diameter. While I'm quite proficient in the mechanical side the use of computer vision is totally new to me and I would be grateful for some help to get me going in the right direction. The image shows a typical example of the stones to be identified by opencv ( the largest stone is about 150mm x 200mm). So far I have only tried SimpleBlobDetector without much success ( it mainly detected the shadows). Bearing in mind that shadows change throughout the day and colours change with weather and season, where do I start? One idea is to use stereo cameras and try to identify object proud of the ground.

Thanks.examples of stones

edit retag flag offensive close merge delete

Comments

In my honest opinion, stereo will be much better then any segmentation/contour/shape based analysis, mainly because you cannot use the colour information as a decent source of info. So why not build a setup and experiment with that.

StevenPuttemans gravatar imageStevenPuttemans ( 2016-04-07 06:17:20 -0500 )edit

Another thing that you could investigate is the use of texture filters, to try to seperate the smooth surfaces of rocks compared to the soil, but that will be a challenging job also!

StevenPuttemans gravatar imageStevenPuttemans ( 2016-04-07 06:18:10 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
5

answered 2016-04-14 02:51:57 -0500

pklab gravatar image

Common stones are colourless,looks dull,gray... it's expected they have low Saturation in HSV. Shadow is dark than... low brightness.

I've made my attempt around above points using MORPH_CLOSE to improve dark areas. This is the result, below is description and code.

Result

Steps:

  1. Scale down the image because it's too big...
  2. Use Gaussian Blur to reduce variation a bit
  3. Convert to HSV
  4. Use MORPH_CLOSE on Saturation channel (use a structure half than wanted stone)
  5. Normalize the histogram to get more stable threshold
  6. Threshold dark saturation to select stones and shadows

Stones and shadows mask

The mask includes shadows. If this it's a problem you could try to remove shadows using the brightness channels... (shadows are dark)

  1. Use MORPH_CLOSE on brightness Channel (use same a structure as above)
  2. Normalize the histogram to get more stable threshold
  3. Threshold dark brightness to select just shadows

shadows mask

Get binary mask for the stone and process it with contour or blobs

  1. Make logical (stones+shadows) AND (NOT shadows) you should obtain just stones
  2. Find contours and process them

This is the code. Get BrightnessAndContrastAuto here.

#define CL_BLU      cv::Scalar(255, 0,      0   )
#define CL_GREEN    cv::Scalar(0,   255,    0   )
#define CL_RED      cv::Scalar(0,   0,      255 )
#define CL_WHITE    cv::Scalar(255, 255,    255 )
const string winName="Stones";
Mat src,dst;
int minSizeMM,thStone,thShadow,removeShadow;
float pix2mm;

//helper function
void MorphClose(const Mat &imgIn,Mat &imgOut,int minThickess=2);
//get the source [here](http://answers.opencv.org/question/75510/how-to-make-auto-adjustmentsbrightness-and-contrast-for-image-android-opencv-image-correction/)
void BrightnessAndContrastAuto(const cv::Mat &src, cv::Mat &dst, float clipHistPercent=0);

// ON TRACK BAR FUNCTION
void onStonesTb(int, void*)
{
  Mat blur,bwStones,bwShadow;
  vector<vector<Point> > contours;
  char buf[80];

  cv::GaussianBlur(src,blur,Size(),2,2);

  // convert to HSV
  Mat src_hsv,brightness,saturation;
  vector<Mat> hsv_planes;
  cvtColor(blur, src_hsv, COLOR_BGR2HSV);
  split(src_hsv, hsv_planes);
  saturation = hsv_planes[1];
  brightness = hsv_planes[2];

  int minSizePx = cvRound(minSizeMM/pix2mm);
  int closerSize = minSizePx /2.0;

  //SELECT STONES (INCLUDING SHADOW)
  MorphClose(saturation,saturation,closerSize);
  BrightnessAndContrastAuto(saturation,saturation,1);
  threshold(saturation,bwStones,thStone,255,THRESH_BINARY_INV); //Get 0..thStone
  //show the selection
  findContours(bwStones, contours, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE);
  for (int i = 0; i < contours.size(); i++)
    cv::drawContours(saturation,contours,i,CL_WHITE,2);
  imshow("Threshold Stones+Shadow on Saturation",saturation);

  if(removeShadow)
  {
    //SELECT DARK AREAS (MAYBE SHADOWS)
    MorphClose(brightness,brightness,closerSize);
    BrightnessAndContrastAuto(brightness,brightness,1);
    threshold(brightness,bwShadow,thShadow,255,THRESH_BINARY); //Get thShadow..255
    //show the selection
    contours.clear();
    findContours(bwShadow, contours, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
    for (int i = 0; i < contours.size(); i++)
      cv::drawContours(brightness,contours,i,CL_WHITE,2);
    imshow("Threshold Shadow on Brightness",brightness);

    //remove shadows from stones
    cv::bitwise_and(bwStones,bwShadow,bwStones);
  }

  //show the result
  src.copyTo(dst);
  findContours(bwStones, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
  Point2f centroid;
  for (int i = 0; i < contours.size(); i++)
  {

    //draw the contour
    polylines(dst, contours[i], true, CL_GREEN,2);
    //get the bounding rect
    cv::Rect rect = cv::boundingRect(contours[i]);
    //ignore small objects
    if ( max(rect.width,rect.width) < minSizePx)
      continue;

    //calculate moments
    cv::Moments M = cv::moments(contours[i], false);
    //reject if area is 0
    double ...
(more)
edit flag offensive delete link more

Comments

What a great post! And what a nice example of the community trying to find a solution for a user. Are you willing of changing this post into a tutorial on the official docs?

StevenPuttemans gravatar imageStevenPuttemans ( 2016-04-14 03:59:12 -0500 )edit

@StevenPuttemans thanks for the appreciation, I'll be honoured to be on the official docs, who/how can do this ? sure after some English improvement :)

pklab gravatar imagepklab ( 2016-04-14 04:18:47 -0500 )edit

@pklab take a look at this recent tutorial PR which gives you an idea on what is needed :)

StevenPuttemans gravatar imageStevenPuttemans ( 2016-04-14 04:40:00 -0500 )edit
1

@pklab, thanks for your contribution and the code example is very much appreciated. This is just what I thought opencv could do. This is a great start to my project.

egdaves gravatar imageegdaves ( 2016-04-15 02:19:26 -0500 )edit
1

You are welcome... do you have some other images to test the code ? if my answer works you could accept it.

BTW a stereo vision would be more reliable and it could help robot navigation too.

pklab gravatar imagepklab ( 2016-04-15 10:43:10 -0500 )edit

@pklab do you think it can lead with shadows in leaf segmentation?

Tarcisioflima gravatar imageTarcisioflima ( 2018-06-25 11:01:57 -0500 )edit

You could first remove the shadows using shadow removal technique.

StevenPuttemans gravatar imageStevenPuttemans ( 2018-07-04 03:12:38 -0500 )edit

@StevenPuttemans I tried some shadow removal technique, but without success.

Tarcisioflima gravatar imageTarcisioflima ( 2018-07-04 09:36:20 -0500 )edit

I tried some shadow removal technique, but without success.

Tarcisioflima gravatar imageTarcisioflima ( 2018-07-04 09:36:30 -0500 )edit
0

answered 2016-04-06 19:55:44 -0500

Tetragramm gravatar image

I suggest using contours. Take a look at the tutorial here, then others on the subject. The best thing is most likely to find the contours and keep the ones that are closed (as in, they come back to the starting point).

If the tutorial doesn't make sense, go back to earlier tutorials. The Core and ImgProc tutorials follow a pretty nice progression for learning OpenCV and image processing.

edit flag offensive delete link more

Comments

Not want to disappoint you, but findContours highly depends on the initial edge map or binary image that is passed to it to find decent contours. Since the stones are less destinctive than the background, this won't be as easy as it sounds. Just did some initial tests, getting the stones is quite challenging!

StevenPuttemans gravatar imageStevenPuttemans ( 2016-04-07 06:14:29 -0500 )edit
1

Thank for suggestions. I'm working on them but everything is slow a I learn.

egdaves gravatar imageegdaves ( 2016-04-12 16:48:12 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2016-04-06 15:26:16 -0500

Seen: 1,009 times

Last updated: Apr 14 '16