Extracting Curved Contours

asked 2020-05-19 03:30:54 -0500

YashRunwal gravatar image

updated 2020-05-20 00:48:27 -0500

So I have been experimenting with OpenCV and extracting a specific portion from an image with python.

My test image is:

enter image description here

From this image, I want to draw the contours around that stockpile of soil as follows:

(Note: This has been done in Paint)

enter image description here

I have tried following things:

  1. Reading the Image and converting it into Grayscale
  2. Morphological Transformation and thresholding as follows:

But I am not sure what to do next? How to extract the aforementioned part of the image?

import cv2

img = cv2.imread('Feature Extraction/soil_stockpile_test_image.png', 
blur = cv2.GaussianBlur(img, (1, 1), 2)
h, w = img.shape[:2]

# Morphological gradient

kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7, 7))
gradient = cv2.morphologyEx(blur, cv2.MORPH_GRADIENT, kernel)
cv2.imshow('Morphological gradient', gradient)

thresh = cv2.threshold(img,150, 255,cv2.THRESH_BINARY_INV)[1]
cnts, h = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cv2.imshow('thresh', thresh)

I tried the histogram back projection method. It somewhat worked but it still shows some unwanted part of the image. I just want the stockpile to be shown in the result.

Here is the code and the resulting image:

# Original Image
orig_img = cv2.imread('Feature Extraction/soil_stockpile_test_image.png')
roi = cv2.imread('Feature Extraction/roi_soil.png')

# HSV Image
hsv_img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2HSV)
hsv_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)

# Region of Interest
roi_hist = cv2.calcHist([hsv_roi], [0, 1], None, [180, 256], [0, 180, 0, 256])
mask = cv2.calcBackProject([hsv_img], [0, 1], roi_hist, [0, 180, 0, 256], 1)

# Remove Noise
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
mask = cv2.filter2D(mask, -1, kernel)
_, mask = cv2.threshold(mask, 210, 255, cv2.THRESH_BINARY)

# Merge and perform bitwise operation
mask = cv2.merge((mask, mask, mask))
result = cv2.bitwise_and(orig_img, mask)

#cv2.imshow('HSV Image', hsv_img)
#cv2.imshow('Region of Interest', roi)
#cv2.imshow('Mask', mask)
cv2.imshow('Result', result)


enter image description here

Edit 1:

After using the grab cut algorithm, with the following code, this is the result I obtained:

# Grab Cut Mask
img = cv2.imread('Feature Extraction/soil_stockpile_test_image.png')

# These params have been hardcoded. Need to be changed for every image.
top_left_x = 300
top_left_y = 150
bottom_right_x = 600
bottom_right_y = 350
Green_color = (0, 255, 0)

# Draw the rectangle
output = cv2.rectangle(img,
                (top_left_x, top_left_y),
                (bottom_right_x, bottom_right_y),
                color=Green_color, thickness=3)

#cv2.imwrite('Output_Image_with_Rectangle.jpg', output)
#cv2.imshow('Output Image with Rectangle', output)

# Grab Cut Mask
mask = np.zeros(img.shape[:2],np.uint8)

bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

rect = (top_left_x, top_left_y, bottom_right_x, bottom_right_y)

mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
new_img = img*mask2[:,:,np.newaxis]

cv2.imwrite('Grab_Cut_img.jpg', new_img)

image description

But there is still some black part there. Can I resize this grab cut image? Also, How do i do that? Also, in the image, you can see that there is still some background. How do I suppress this?

I really need your help. Thanks.

edit retag flag offensive close merge delete



looks good. What result would you desire via automation? Would you refine by angularity? Or manual feedback loop (GrabCut mask?). Or is it always in the corner like a watermark logo? Would mask opening remove enough "noise" ?

After you apply your mask, apply Hough Lines to said mask for contour separation. Then find and mask the minimum enclosing rectangle of... ??? The largest area or the contour in a position near x/y. You could expand the canvas with CopyMakeBorder for better estimation?

Depth sensor? Re-take photo? Hire assistant?

kpachinger gravatar imagekpachinger ( 2020-05-19 11:21:59 -0500 )edit

@kpachinger Thanks for your reply but I don't quite understand what you mean by refine by angularity. I have not done this before.

YashRunwal gravatar imageYashRunwal ( 2020-05-19 22:31:33 -0500 )edit

@kpachinger I used the Grab Cut algorithm and the result can be seen in Edit 1 in the question. But there is still some background in the mask. How do i supress that?

YashRunwal gravatar imageYashRunwal ( 2020-05-20 00:28:09 -0500 )edit

You should GC_INIT_WITH_MASK, and your mask needs correct GC values. Initial progress from mask is your intent as opposed to a uniform rect area.

Consider that GrabCut is now expected to discern a pile from similar nearby features. It will be determined by rgba area continuity, and provided mask values.

Accurate and consistent marks on GC mask will affect "success". The benefit of defining the pile at it's angle or enclosing rectangle is a better in/out.

The question is what degree of correct value can you give? Without manually negating the purpose?

kpachinger gravatar imagekpachinger ( 2020-05-20 05:07:09 -0500 )edit

So you mean I should replace GC_INIT_WITH_RECT with GC_INIT_WITH_MASK and use the mask I had created previously and not the mask created for the Grabcut? But If I do that I get the follwing error: (-215:Assertion failed) !bgdSamples.empty() && !fgdSamples.empty() in function 'initGMMs'

YashRunwal gravatar imageYashRunwal ( 2020-05-20 08:00:27 -0500 )edit

Yes. The GrabCut accepts 4 specific mat values. They are assigned by name or number, and are not a linear range but a code.

A pixel set to GC_FGD or GC_BGD is decided, and it affects neighboring pixels less than PR_FGD and PR_BGD when GrabCut considers foreground/background. That is my experience.

It is possible that GrabCut completes without certainty of foreground/background. Consider an image of evenly distributed noise. The resulting mask, white or black, would impact later outcomes. Would Grabcut flip a coin or should you personally handle it?

It is possible to use a pipe for the init, to allow MASK | RECT. This allows more logic and less critical errors. Perhaps you put mask/grabcut through a feedback loop and mark the mask multiple times until satisfied?

kpachinger gravatar imagekpachinger ( 2020-05-20 09:09:09 -0500 )edit

@YashRunwal. Comment out np.zeros. U don't needed this.

supra56 gravatar imagesupra56 ( 2020-05-20 20:43:47 -0500 )edit

@supra56 Do you mean the masks and bg and fgmodel?

YashRunwal gravatar imageYashRunwal ( 2020-05-24 00:30:29 -0500 )edit

I am still not able to 100% solve this. Any help is appreciated. Thank You.

YashRunwal gravatar imageYashRunwal ( 2020-05-24 00:30:57 -0500 )edit