Francesco Sgaramella's profile - activity

I have done a random drawing on a whiteboard and NAO robot has taken a picture and tried to re-create the same drawing.

My drawing:

NAO's drawing:

At this point I would like to write some conclusions about it, specifically I want to extract the contours from both pictures and match the contours using the OpenCV function cv2.matchShapes().

However, I wrote a small Python code script for this and it gives me some errors. Here is the code:

import numpy as np
import cv2

#get the pictures from the forlder
original = cv2.imread('eightgon.jpg')
drawn = cv2.imread('eightgon1.jpg')

#make them gray    
originalGray = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)
drawnGray = cv2.cvtColor(drawn, cv2.COLOR_BGR2GRAY)

#apply erosion
kernel = np.ones((2, 2),np.uint8)
originalErosion = cv2.erode(originalGray, kernel, iterations = 1)
drawnErosion = cv2.erode(drawnGray, kernel, iterations = 1)

#retrieve edges with Canny
thresh = 175
originalEdges = cv2.Canny(originalErosion, thresh, thresh*2)
drawnEdges = cv2.Canny(drawnErosion, thresh, thresh*2)

#extract contours
originalContours, Orighierarchy = cv2.findContours(originalEdges, cv2.cv.CV_RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
drawnContours, Drawnhierarchy = cv2.findContours(drawnEdges, cv2.cv.CV_RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)

print cv2.matchShapes(drawnContours,originalContours,cv2.cv.CV_CONTOURS_MATCH_I1, 0.0)

When I run this simple code, it returns me this error:

File "C:/Python27/getResults.py", line 32, in <module>
    ret = cv2.matchShapes(drawnContours,originalContours,cv2.cv.CV_CONTOURS_MATCH_I1, 0.0)
TypeError: contour1 is not a numpy array, neither a scalar

Since the error tells me that the contours should be arrays.. I make a slight change in the code like this:

cnt1 = np.asarray(drawnContours, np.int0)
cnt2 = np.asarray(originalContours, np.int0)
print cv2.matchShapes(cnt1,cnt2,cv2.cv.CV_CONTOURS_MATCH_I1, 0.0)

and in this case it returns me this error: ValueError: setting an array element with a sequence.

What am I doing wrong? Any help is apreciated!

I implemented a Python script for recognizing shapes in hand drawings. However, the script recognizes more shapes than needed.

Here is an example picture:

and this is the output of the script:

Part of the code I wrote is the following:

def create_graph(vertex, color):
    for g in range(0, len(vertex)-1):
        for y in range(0, len(vertex[0][0])-1):
            cv2.circle(newimg, (vertex[g][0][y], vertex[g][0][y+1]), 3, (255,255,255), -1)
            cv2.line(newimg, (vertex[g][0][y], vertex[g][0][y+1]), (vertex[g+1][0][y], vertex[g+1][0][y+1]), color, 2)
    cv2.line(newimg, (vertex[len(vertex)-1][0][0], vertex[len(vertex)-1][0][1]), (vertex[0][0][0], vertex[0][0][1]), color, 2)


img = cv2.imread('star.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

#Remove of noise, if any
kernel = np.ones((2, 2),np.uint8)
erosion = cv2.erode(gray, kernel, iterations = 1)

#Create a new image of the same size of the starting image
height, width = gray.shape
newimg = np.zeros((height, width, 3), np.uint8)

#Canny edge detector
thresh = 175
edges = cv2.Canny(erosion, thresh, thresh*2)

contours,hierarchy = cv2.findContours(edges, cv2.cv.CV_RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
for b,cnt in enumerate(contours):
    if hierarchy[0,b,3] == -1: #<-the mistake might be here
       approx = cv2.approxPolyDP(cnt,0.015*cv2.arcLength(cnt,True), True)
       clr = (255, 0, 0)
       create_graph(approx, clr) #function for drawing the found contours in the new img
cv2.imwrite('starg.jpg', newimg)

I did not post all the code because is useless. I think that I am mistaking the use of hierarchy for finding the contours. I am not such a Python expert and I did not understand that well the use of hierarchy in the contours. Does anybody have suggestions?

Hello people, I am a BEGINNER of Python and I would really need some help. I have a (working) script that retrieves simple shapes in images drawn by hand. The next step is to determine the spatial relations between the shapes and a data structure that contains all those information. I know how to determine the spatial relations between the shapes, like inside, next to, above and below. However, I need a data structure to contain all those information: shapes (x,y) coordinate of the edges and relation with other shapes. Each shape should contain the info.

Shape = (x0, y0, ..., xn, yn), position(Shape)

where

• (x0, y0, ..., xn, yn) are the coordinates of the edges of the shape
• position(Shape) is the relation of the shape against another.

For example:

SQUARE1: (x0, y0, x1, y1, x2, y2, x3, y3), below(T1)
TRIANGLE1: (x0, y0, x1, y1, x2, y2), above(S1)
CIRCLE1: (radius), inside(S1), above(C2)
CIRCLE2: (radius), inside(S1), below(C1)

But how to implement it in python? I read about dictionaries here but I am not sure if it's the right tool to use here. Why? Because I need to write in the data structure the position of a shape against others. So how could I write this relation? I am quite confused about the argument and I would really need an help.

I am having a lot of troubles in python programming, mostly because I am a beginner.

I first applied findContours to a picture and I have to save the retrieved shape corners and the shapes centroids in data structure for latter modifications.

I created a data structure for the shapes that looks like this (the number of shapes is not known):

Shapes = [[shape1-Corners],
          [shape2-Corners],
          [shape3-Corners],
          [...]
          [shapeN-Corners]]

and another one containing the centers (again the number of shapes is not known):

Centers = [x0, y0, x1, y1, x2, y2,...,xn, yn]

Now I have to access the data in those two structures and perform the pointPolygonTest to check if a shape is inside another one, but OF COURSE it's really hard (because I understand that this solution with two data structure is not really the best)

That's the code I wrote for accessing the data structure:

for k in range(0, len(centers), 2):
    nowcx = centers[k]
    nowcy = centers[k+1]
    for j in range(len(shapes)):
        if cv2.pointPolygonTest(shapes[j],(nowcx, nowcy),False)==1:
            print "inside"
        else:
            print "outside"

For easier understanding, this is the picture with the centroids in red. You can see that thare are 3 shapes: triangle, square and pentagon. So the data structures contain 3 records for the 3 shapes and 6 elements for the 3 centroids.

https://www.dropbox.com/s/vibl0h0y4ses1za/graph.png

The output gives me 9 comparisons, when they should be actually 3:

1. triangle against square -> outside
2. triangle against pentagon -> inside
3. square againstr pentagon -> inside

That's the output:

inside
outside
inside
outside
inside
inside
inside
outside
inside

What am I doing wrong? Please help me. Thanks in advance!!

I was not gifted like you

how can I state if a centerpoint is above (or below, next to, inside) another? what should be the condition?

Hello people, I am here to ask for an advice because, after surfing the web, I could not find myself a strategy.

I have to implement a script in python that states which are the spatial relations between objects (the objects are shapes).

So far I have a script that recognizes shapes in images using findContours() function and now I have to state which are the spatial relations between the object.

An example: I got that a is a triangle and b is a square, what is the realtion between a and b? Inside, Above, Below, NextTo, etc. I do this in order to obtain a graph containing the shapes and the spatial relations between them.

How could I do that? Which strategy would you advice me? Or, which package of python fits better the requirements?

Thanks in advance for your help!

Thank you so much! What you just give me is a great starting point! I will check the subPixel and the Canny. Thanks again! I will update the post as soon as I get something more. Could be helpful for others.

I'm working on a project and at some point I have to detect corners and lines in a hand drawn picture. I am doing the work in Python but the result I get is not that good.

The code I wrote is the following:

import sys
from math import sin, cos, sqrt, pi
import cv
src = cv.LoadImage('draw.jpg')
cv.NamedWindow("Source", 1)
cv.NamedWindow("Result", 1)
dst = cv.CreateImage(cv.GetSize(src), 8, 1)
color_dst = cv.CreateImage(cv.GetSize(src), 8, 3)
storage = cv.CreateMemStorage(0)
lines = 0
cv.Canny(src, dst, 1.0, 4.0, aperture_size=3)
cv.CvtColor(dst, color_dst, cv.CV_GRAY2BGR)
lines = cv.HoughLines2(dst, storage, cv.CV_HOUGH_PROBABILISTIC, 1, pi / 180, 10, 0, 10)
for line in lines:
    cv.Line(color_dst, line[0], line[1], cv.CV_RGB(0, 255, 0), 2, 8)
cornerMap = cv.CreateMat(dst.height, dst.width, cv.CV_32FC1)
cv.CornerHarris(dst,cornerMap,3)
for y in range(0,dst.height):
    for x in range (0, dst.width):
        harris = cv.Get2D(cornerMap, y, x)
        if harris[0] >10e-06:
            cv.Circle(color_dst, (x,y),2,cv.RGB(115,0,25))
cv.ShowImage("Source", src)
cv.ShowImage("Hough", color_dst)
cv.SaveImage("draw_det.png", color_dst)
cv.WaitKey()

The input picture can be seen here:

link text

While the resultant picture can be seen here:

link text

The result is not what I am expecting. What I want as output is to see the corner points detected in red and the lines connecting them in green. While what I get is something that does not make sense by looking at the pictures. What am I doing wrong?