Face detection: how to stop detecting multiple mouths in real time OpenCV Java
I'm writing a java program using OpenCV that opens the webcam and detects the face, eyes and mouth in real time. Everything works fine except for the mouth detection as it seems to be detecting many mouths all over the face and also appears to be detecting the eyes as mouths. I was wondering why this might be and how I could fix it.
I was also wondering if there was a way to set constraints for the cascades e.g. one mouth per face, mouth must be below the eyes etc.
Let me know if you want the rest of the code.
Code for this class:
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc;
import org.opencv.objdetect.CascadeClassifier;
public class FaceProcessor {
private CascadeClassifier faceCascade;
private CascadeClassifier eyesCascade;
private CascadeClassifier mouthCascade;
public FaceProcessor() {
faceCascade = new CascadeClassifier(
FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath());
eyesCascade = new CascadeClassifier(
FaceDetector.class.getResource("haarcascade_eye_tree_eyeglasses.xml").getPath());
mouthCascade = new CascadeClassifier(FaceDetector.class.getResource("Mouth.xml").getPath());
if (faceCascade.empty()) {
System.out.println("Face cascade failed to load.");
return;
} else {
System.out.println("Face cascade loaded successfully.");
}
if (eyesCascade.empty()) {
System.out.println("Eyes cascade failed to load.");
} else {
System.out.println("Eyes cascade loaded successfully.");
}
if (mouthCascade.empty()) {
System.out.println("Mouth cascade failed to load.");
} else {
System.out.println("Mouth cascade loaded successfully.");
}
}
public Mat detect(Mat inputframe) {
Mat mRgba = new Mat();
Mat mGrey = new Mat();
MatOfRect faces = new MatOfRect();
inputframe.copyTo(mRgba);
inputframe.copyTo(mGrey);
Imgproc.cvtColor(mRgba, mGrey, Imgproc.COLOR_BGR2GRAY);
Imgproc.equalizeHist(mGrey, mGrey);
faceCascade.detectMultiScale(mGrey, faces);
System.out.println(String.format("Detected %s face(s)", faces.toArray().length));
faceCascade.detectMultiScale(mGrey, faces, 1.1, 2, 0, new Size(30, 30), new Size());
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++) {
Point centre1 = new Point(facesArray[i].x + facesArray[i].width * 0.5,
facesArray[i].y + facesArray[i].height * 0.5);
Core.ellipse(mRgba, centre1, new Size(facesArray[i].width * 0.5, facesArray[i].height * 0.5), 0, 0, 360,
new Scalar(192, 202, 235), 4, 8, 0);
Mat faceROI = mGrey.submat(facesArray[i]);
MatOfRect eyes = new MatOfRect();
eyesCascade.detectMultiScale(faceROI, eyes, 1.1, 2, 0, new Size(30, 30), new Size());
Rect[] eyesArray = eyes.toArray();
for (int j = 0; j < eyesArray.length; j++) {
Point centre2 = new Point(facesArray[i].x + eyesArray[j].x + eyesArray[j].width * 0.5,
facesArray[i].y + eyesArray[j].y + eyesArray[j].height * 0.5);
int radius = (int) Math.round((eyesArray[j].width + eyesArray[j].height) * 0.25);
Core.circle(mRgba, centre2, radius, new Scalar(255, 202, 121), 4, 8, 0);
}
MatOfRect mouth = new MatOfRect();
mouthCascade.detectMultiScale(faceROI, mouth, 1.1, 2, 0, new Size(30, 30), new Size());
Rect[] mouthArray = mouth.toArray();
for (int k = 0; k < mouthArray.length; k++) {
Point centre3 = new Point(facesArray[i].x + mouthArray ...
you can increase the 'minNeighbours' param from 2 (very low) to maybe 5 or such, to weed out more false detections.
@berak I tried that but it's still detecting mouths on the eyes. Only if I change it to a really high number like 100 does it work - but even then it's not perfect.
IMHO you can recalculate
faceROI
for mouth detection@sturkmen How would I do that? Would you mind writing some example code?
let me try :) i am not familiar with java. wait
here is my trial addition
as i said i can't try the code . but i think it should work. please try it.
@sturkmen for these lines
facesArray[i].y = facesArray[i].y + facesArray[i].height * 0.5; facesArray[i].height = facesArray[i].height * 0.5;
it says there's a type mismatch: cannot convert from double to int?try
facesArray[i].y = facesArray[i].y + round(facesArray[i].height * 0.5);
facesArray[i].height = round(facesArray[i].height * 0.5);
@sturkmen it still says "The method round(double) is undefined for the type FaceProcessor"