Ask Your Question
2

Face detection: how to stop detecting multiple mouths in real time OpenCV Java

asked 2015-08-20 12:28:41 -0500

inara gravatar image

I'm writing a java program using OpenCV that opens the webcam and detects the face, eyes and mouth in real time. Everything works fine except for the mouth detection as it seems to be detecting many mouths all over the face and also appears to be detecting the eyes as mouths. I was wondering why this might be and how I could fix it.

I was also wondering if there was a way to set constraints for the cascades e.g. one mouth per face, mouth must be below the eyes etc.

Let me know if you want the rest of the code.

Code for this class:

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.CascadeClassifier;

public class FaceProcessor {
private CascadeClassifier faceCascade;
private CascadeClassifier eyesCascade;
private CascadeClassifier mouthCascade;

public FaceProcessor() {
faceCascade = new CascadeClassifier(
        FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath());
eyesCascade = new CascadeClassifier(
        FaceDetector.class.getResource("haarcascade_eye_tree_eyeglasses.xml").getPath());
mouthCascade = new CascadeClassifier(FaceDetector.class.getResource("Mouth.xml").getPath());

if (faceCascade.empty()) {
    System.out.println("Face cascade failed to load.");
    return;
} else {
    System.out.println("Face cascade loaded successfully.");
}

if (eyesCascade.empty()) {
    System.out.println("Eyes cascade failed to load.");
} else {
    System.out.println("Eyes cascade loaded successfully.");
}

if (mouthCascade.empty()) {
    System.out.println("Mouth cascade failed to load.");
} else {
    System.out.println("Mouth cascade loaded successfully.");
}

}

public Mat detect(Mat inputframe) {
Mat mRgba = new Mat();
Mat mGrey = new Mat();
MatOfRect faces = new MatOfRect();
inputframe.copyTo(mRgba);
inputframe.copyTo(mGrey);
Imgproc.cvtColor(mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);
Imgproc.equalizeHist(mGrey, mGrey);
faceCascade.detectMultiScale(mGrey, faces);
System.out.println(String.format("Detected %s face(s)", faces.toArray().length));
faceCascade.detectMultiScale(mGrey, faces, 1.1, 2, 0, new Size(30, 30), new Size());
Rect[] facesArray = faces.toArray();

for (int i = 0; i < facesArray.length; i++) {
    Point centre1 = new Point(facesArray[i].x + facesArray[i].width * 0.5,
            facesArray[i].y + facesArray[i].height * 0.5);
    Core.ellipse(mRgba, centre1, new Size(facesArray[i].width * 0.5, facesArray[i].height * 0.5), 0, 0, 360,
            new Scalar(192, 202, 235), 4, 8, 0);

    Mat faceROI = mGrey.submat(facesArray[i]);
    MatOfRect eyes = new MatOfRect();

    eyesCascade.detectMultiScale(faceROI, eyes, 1.1, 2, 0, new Size(30, 30), new Size());

    Rect[] eyesArray = eyes.toArray();

    for (int j = 0; j < eyesArray.length; j++) {
        Point centre2 = new Point(facesArray[i].x + eyesArray[j].x + eyesArray[j].width * 0.5,
                facesArray[i].y + eyesArray[j].y + eyesArray[j].height * 0.5);
        int radius = (int) Math.round((eyesArray[j].width + eyesArray[j].height) * 0.25);
        Core.circle(mRgba, centre2, radius, new Scalar(255, 202, 121), 4, 8, 0);
    }

    MatOfRect mouth = new MatOfRect();

    mouthCascade.detectMultiScale(faceROI, mouth, 1.1, 2, 0, new Size(30, 30), new Size());

    Rect[] mouthArray = mouth.toArray();

    for (int k = 0; k < mouthArray.length; k++) {
        Point centre3 = new Point(facesArray[i].x + mouthArray ...
(more)
edit retag flag offensive close merge delete

Comments

you can increase the 'minNeighbours' param from 2 (very low) to maybe 5 or such, to weed out more false detections.

berak gravatar imageberak ( 2015-08-20 12:35:09 -0500 )edit

@berak I tried that but it's still detecting mouths on the eyes. Only if I change it to a really high number like 100 does it work - but even then it's not perfect.

inara gravatar imageinara ( 2015-08-20 12:37:44 -0500 )edit

IMHO you can recalculate faceROI for mouth detection

sturkmen gravatar imagesturkmen ( 2015-08-20 13:16:11 -0500 )edit

@sturkmen How would I do that? Would you mind writing some example code?

inara gravatar imageinara ( 2015-08-20 14:15:44 -0500 )edit

let me try :) i am not familiar with java. wait

sturkmen gravatar imagesturkmen ( 2015-08-20 14:19:46 -0500 )edit

here is my trial addition

    MatOfRect mouth = new MatOfRect();
    // insert this lines
    facesArray[i].y = facesArray[i].y + facesArray[i].height * 0.5;
    facesArray[i].height = facesArray[i].height * 0.5;

    faceROI = mGrey.submat(facesArray[i]);

    mouthCascade.detectMultiScale(faceROI, mouth, 1.1, 2, 0, new Size(30, 30), new Size());
sturkmen gravatar imagesturkmen ( 2015-08-20 14:26:08 -0500 )edit

as i said i can't try the code . but i think it should work. please try it.

sturkmen gravatar imagesturkmen ( 2015-08-20 14:29:02 -0500 )edit

@sturkmen for these lines facesArray[i].y = facesArray[i].y + facesArray[i].height * 0.5; facesArray[i].height = facesArray[i].height * 0.5; it says there's a type mismatch: cannot convert from double to int?

inara gravatar imageinara ( 2015-08-20 17:56:49 -0500 )edit
  • try

    facesArray[i].y = facesArray[i].y + round(facesArray[i].height * 0.5);

    facesArray[i].height = round(facesArray[i].height * 0.5);

sturkmen gravatar imagesturkmen ( 2015-08-20 19:42:43 -0500 )edit

@sturkmen it still says "The method round(double) is undefined for the type FaceProcessor"

inara gravatar imageinara ( 2015-08-21 05:20:43 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
-1

answered 2015-08-21 07:01:43 -0500

you can recalculate faceROI before mouth detection as shown below.

MatOfRect mouth = new MatOfRect();
// insert this lines
facesArray[i].height = (int) Math.round(facesArray[i].height * 0.5);
facesArray[i].y = facesArray[i].y + facesArray[i].height;

faceROI = mGrey.submat(facesArray[i]);

mouthCascade.detectMultiScale(faceROI, mouth, 1.1, 2, 0, new Size(30, 30), new Size());
edit flag offensive delete link more

Comments

This still shows the red rectangle on the top of the eyebrow for mouth

devdroid6 gravatar imagedevdroid6 ( 2016-10-13 05:03:00 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2015-08-20 12:28:41 -0500

Seen: 1,616 times

Last updated: Aug 21 '15