Revision history [back]

Get ROI from face landmark points cv2 dlib

Am using following code to draw facial landmark points using dlib , on to the frames captured from webcam in realtime, now what am looking for is to get the ROI that bounds all the points complying the face , the code is as follows:

import cv2
import dlib
import numpy
from imutils.video import FPS
from imutils.video import WebcamVideoStream
import imutils

PREDICTOR_PATH = "./shape_predictor_68_face_landmarks.dat"
predictor = dlib.shape_predictor(PREDICTOR_PATH)
cascade_path = 'cascade/haarcascade_frontalface_default.xml'
cascade = cv2.CascadeClassifier(cascade_path)

webcam = WebcamVideoStream(src=0).start()
fps = FPS().start()

while True:
    im = webcam.read()
    im = imutils.resize(im, width=400)

    faces = cascade.detectMultiScale(im, 1.3, 5)
    if len(faces) != 0:
        for (x, y, w, h) in faces.astype(long):
            rect = dlib.rectangle(x, y, x + w, y + h)
            #cv2.imwrite('face.png', rect)
            get_landmarks = numpy.matrix([[p.x, p.y] for p in predictor(im, rect).parts()])

        for idx, point in enumerate(get_landmarks):
            pos = (point[0, 0], point[0, 1])
            cv2.putText(im, str(idx), pos,
                        fontFace=cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,
                        fontScale=0.4,
                        color=(0, 0, 255))
           # cv2.drawContours(im, [pos], -1, (0, 255, 0), 2)
#            hullIndex = cv2.convexHull(pos, returnPoints=False)
#            cv2.imwrite('face.png', hullIndex)
            cv2.circle(im, pos, 3, color=(0, 255, 255))
    cv2.imshow('Result', im)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
    fps.update()

cv2.destroyAllWindows()

I want to dump the facial landmark points into image file. but am unable to understand , how I can extract ROI from these landmark points and then dump to file using

cv2.imwrite('face.png', hullIndex)

, but here I don't want the rectangle drwaned using

faces = cascade.detectMultiScale(im, 1.3, 5)

, I want only the shape that circumfencec the facial landmark points, the output of the above code looks like below.

So, just dump only points that circumferences the facial edge points.

One proposed solution for this is given below, but it's also not working.

After getting the 68 facial landmarks from the predictor, You may simply iterate over all the points and update the minX, minY, maxX, maxY which after the iteration would yield the bounding rect exactly enclosing all the facial landmarks.

You may also use cv2.boundingRect(points) but you need to transform all the points to a numpy array before passing to this method.

But I would recommend you the first solution and it is fairly easy to implement as

minX, minY = 10000000, 10000000
maxX, maxY = 0, 0
for point in get_landmarks:
    if point[0] < minX:
        minX = point[0]
    elif point[0] > maxX:
        maxX = point[0]
    elif point[1] < minY:
        minY = point[1]
    elif point[1] > maxY:
        maxY = point[1]

bounding_rect = [minX, minY, maxX - minX, maxY - minY]