Get ROI from face landmark points cv2 dlib

asked 2017-01-08 04:23:40 -0500

U.Swap gravatar image

updated 2017-01-08 04:25:42 -0500

Am using following code to draw facial landmark points using dlib , on to the frames captured from webcam in realtime, now what am looking for is to get the ROI that bounds all the points complying the face , the code is as follows:

import cv2
import dlib
import numpy
from imutils.video import FPS
from imutils.video import WebcamVideoStream
import imutils

PREDICTOR_PATH = "./shape_predictor_68_face_landmarks.dat"
predictor = dlib.shape_predictor(PREDICTOR_PATH)
cascade_path = 'cascade/haarcascade_frontalface_default.xml'
cascade = cv2.CascadeClassifier(cascade_path)

webcam = WebcamVideoStream(src=0).start()
fps = FPS().start()

while True:
    im = webcam.read()
    im = imutils.resize(im, width=400)

    faces = cascade.detectMultiScale(im, 1.3, 5)
    if len(faces) != 0:
        for (x, y, w, h) in faces.astype(long):
            rect = dlib.rectangle(x, y, x + w, y + h)
            #cv2.imwrite('face.png', rect)
            get_landmarks = numpy.matrix([[p.x, p.y] for p in predictor(im, rect).parts()])

        for idx, point in enumerate(get_landmarks):
            pos = (point[0, 0], point[0, 1])
            cv2.putText(im, str(idx), pos,
                        fontFace=cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,
                        fontScale=0.4,
                        color=(0, 0, 255))
           # cv2.drawContours(im, [pos], -1, (0, 255, 0), 2)
#            hullIndex = cv2.convexHull(pos, returnPoints=False)
#            cv2.imwrite('face.png', hullIndex)
            cv2.circle(im, pos, 3, color=(0, 255, 255))
    cv2.imshow('Result', im)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    fps.update()

cv2.destroyAllWindows()

I want to dump the facial landmark points into image file. but am unable to understand , how I can extract ROI from these landmark points and then dump to file using

cv2.imwrite('face.png', hullIndex)

, but here I don't want the rectangle drwaned using

faces = cascade.detectMultiScale(im, 1.3, 5)

, I want only the shape that circumfencec the facial landmark points, the output of the above code looks like below.

enter image description here

So, just dump only points that circumferences the facial edge points.

One proposed solution for this is given below, but it's also not working. Infact a confusing for me, a bit elaboration for this might help me understand it!

After getting the 68 facial landmarks from the predictor, You may simply iterate over all the points and update the minX, minY, maxX, maxY which after the iteration would yield the bounding rect exactly enclosing all the facial landmarks.

You may also use cv2.boundingRect(points) but you need to transform all the points to a numpy array before passing to this method.

But I would recommend you the first solution and it is fairly easy to implement as

minX, minY = 10000000, 10000000
maxX, maxY = 0, 0
for point in get_landmarks:
    if point[0] < minX:
        minX = point[0]
    elif point[0] > maxX:
        maxX = point[0]
    elif point[1] < minY:
        minY = point[1]
    elif point[1] > maxY:
        maxY = point[1]

bounding_rect = [minX, minY, maxX - minX, maxY - minY]
edit retag flag offensive close merge delete

Comments

"but it's also not working." -- sorry, but that is never helpful at all.

we can only help you, if you give a better explanation of the problem you're facing

berak gravatar imageberak ( 2017-01-08 04:44:54 -0500 )edit

btw, also the for idx, point in enumerate(get_landmarks): loop needs another indent, there might be more than 1 face in your image.

berak gravatar imageberak ( 2017-01-08 04:49:09 -0500 )edit

sorry for misleading terms, but can you propose some effective solution for this.

U.Swap gravatar imageU.Swap ( 2017-01-08 05:05:51 -0500 )edit

imho, all you need is already there. once you have the boundingRect, you simply save it as:

cv2.imwrite("my.png", img[minY:maxY, minX:maxX])

again, please be more clear about where exactly your problem is.

berak gravatar imageberak ( 2017-01-08 05:25:32 -0500 )edit

that's the one, thanx. but can you just tell me how can i associate boundingRect, before

cv2.imwrite("my.png", img[minY:maxY, minX:maxX])
U.Swap gravatar imageU.Swap ( 2017-01-08 05:37:27 -0500 )edit

if you used r = cv2.boundingRect(points), then it would be:

cv2.imwrite("my.png", img[r[1]:r[1]+r[3], r[0]:r[0]+r[2])
berak gravatar imageberak ( 2017-01-08 09:00:07 -0500 )edit
1

Thanks, thats what I was looking for.

U.Swap gravatar imageU.Swap ( 2017-01-09 03:03:13 -0500 )edit