Ask Your Question
0

Python OpenCV: Canny Edge Detection for Stickerless Rubik's Cube

asked 2020-05-21 11:35:50 -0500

DWO gravatar image

updated 2020-05-22 08:55:35 -0500

supra56 gravatar image

Hello,

I've started using OpenCV a couple of weeks ago and thought that building a Rubik's cube solver would be a great first project for me. I've done some research and started to implement the color detection for the Rubik's cube. However, everything online uses a Rubik's cube with stickers. This makes sense because the black borders around the stickers make it very easy for edge detection to detect individual squares. But I am using a stickerless Rubik's cube which means that I am struggling to differentiate between the squares due to there being no obvious border.

colors = {
    'red': ([163, 107,  11], [180, 255, 255]),  # Red
    'blue': ([79, 155, 50], [107, 255, 255]),   # Blue
    'yellow': ([17, 58, 50], [34, 255, 255]),  # Yellow
    'orange': ([5, 130, 50], [20, 255, 255]),  # Orange
    'green': ([68, 86, 50], [89, 255, 255]),   # Green
    'white': ([0, 0, 50], [180, 37, 255])      # White
}

hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV);
mask = np.zeros(hsv.shape, dtype=np.uint8)

open_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,7))
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
for color, (lower, upper) in colors.items():
    lower = np.array(lower, dtype=np.uint8)
    upper = np.array(upper, dtype=np.uint8)
    color_mask = cv2.inRange(hsv, lower, upper)
    color_mask = cv2.morphologyEx(color_mask, cv2.MORPH_OPEN, open_kernel, iterations=1)
    color_mask = cv2.morphologyEx(color_mask, cv2.MORPH_CLOSE, close_kernel, iterations=5)

    color_mask = cv2.merge([color_mask, color_mask, color_mask])
    mask = cv2.bitwise_or(mask, color_mask)

mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
hsv = cv2.bitwise_and(hsv, hsv, mask=mask)

canny = cv2.Canny(cv2.GaussianBlur(hsv, (3, 3), 0), 20, 40)

cv2.imshow("Canny Edge", canny);

I've tried playing around with canny thresholds but it just has a lot of trouble with the borders of the squares. Making the edge detection very unstable and unusable.

Does anyone have any tips on how I can improve the edge detection? Should I just buy a cheap Rubik's cube with stickers?

image description

Canny output:

image description

Thanks.

edit retag flag offensive close merge delete

Comments

U merely have 5 colours instead of 6 colours. So how come?

supra56 gravatar imagesupra56 ( 2020-05-21 13:27:18 -0500 )edit

Sorry i was meant to put white in there as well. I have edited the code to fix it.

DWO gravatar imageDWO ( 2020-05-21 13:32:04 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-05-22 04:25:08 -0500

kbarni gravatar image

Can you post an example image? Maybe you can use some contraints if you know the position of the camera/cube.

Anyway, I see a few problems with your approach:

  • you use the hsv image for the edge detection. The hsv image has less contrast than the original one, so it isn't a good starting point. Worse, you'll have false boundaries when the hue for red goes from 179->0.
  • To make things worse, you blur the image to have even less edges...
  • not related, but you should keep color_mask and mask in grayscale (no need to cv2.merge(...) and mask=cv2.cvtcolor(...).

I suggest to apply edge detection (laplacian, canny or sobel) on the color_mask for each color, then merge these "color edges" too. Something like:

...
edges = np.zeros(hsv.shape, dtype=np.float64)
for color, (lower, upper) in colors.items():
    ...
    color_edge = cv2.Laplacian(color_mask,cv2.CV_64F)
    edges = edges + color_edge
    mask = mask + color_mask
edit flag offensive delete link more

Comments

@kbarni. This doesn't work for me.

supra56 gravatar imagesupra56 ( 2020-05-22 08:15:46 -0500 )edit

The reason I blur is that there is noise in the webcam that makes it difficult to detect square contours for the cubes. I have implemented your suggestion of performing edge detection for each color, unfortunately, it does not return great results. My goal for this is to be able to point the webcam at the cube at an angle so that the webcam can see 3 sides of the cube at once and detect the squares on each of those 3 sides. And then I'm going to point another webcam at the opposite end to detect the other 3 sides. But I'm thinking this maybe be more trouble than it's worth with a stickerless cube due to the lack of information that would be available at that angle for the edges of each square.

DWO gravatar imageDWO ( 2020-05-22 09:45:25 -0500 )edit

Yep, this method works best for unsolved cubes.

Anyway, seeing the actual picture... maybe you could try to detect the edges of the cube.This is easy if the cube has fixed position, otherwise use the Harris detector. Once you have the 4 edges of a face, use WarpPerspective to project it to a plane. Then you can get the 9 squares simply by sampling the colors.

kbarni gravatar imagekbarni ( 2020-05-25 05:05:21 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-05-21 11:35:50 -0500

Seen: 62 times

Last updated: May 22