Ask Your Question
2

Is there any limit on maximum number of faces detected using DNN face detector?

asked 2020-01-29 05:26:26 -0500

Amogh gravatar image

updated 2020-01-29 12:37:07 -0500

By using OpenCV version 4.2.0 in c++ (VS 2019) I created project which performs face detection on the given image. I used Opencv's DNN face detector which uses res10_300x300_ssd_iter_140000_fp16.caffemodel model to detect faces.

Program works very well and detects faces as expected, but while playing and trying with different images, I come to know that for some images program does not detect any single face (even frontal faces) on the other hand If I perform dlib's HOG+SVM detector then it detects 46 faces.

Image, in which HOG+SVM detector able to detect 46 faces:

image description

Below is the function which performs face detection using opencv and which detects 0 faces:

//variables which are used in function
const double inScaleFactor = 1.0;
const cv::Scalar meanVal = cv::Scalar(104.0, 177.0, 123.0);
const size_t inWidth = 300;
const size_t inHeight = 300;

std::vector<FaceDetectionResult> namespace_name::FaceDetection::detectFaceByOpenCVDNN(std::string filename, FaceDetectionModel model)
{        

    Net net;        
    cv::Mat frame = cv::imread(filename);
    cv::Mat inputBlob;
    std::vector<FaceDetectionResult> vec;


    if (frame.empty())
        throw std::exception("provided image file is not found or unable to open.");

    int frameHeight = frame.rows;
    int frameWidth = frame.cols;

    if (model == FaceDetectionModel::CAFFE)
    {            
        net = cv::dnn::readNetFromCaffe(caffeConfigFile, caffeWeightFile);
        inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, false, false);
    }
    else
    {            
        net = cv::dnn::readNetFromTensorflow(tensorflowWeightFile, tensorflowConfigFile);
        inputBlob = cv::dnn::blobFromImage(frame, inScaleFactor, cv::Size(inWidth, inHeight), meanVal, true, false);
    }

    net.setInput(inputBlob, "data");
    cv::Mat detection = net.forward("detection_out");

    cv::Mat detectionMat(detection.size[2], detection.size[3], CV_32F, detection.ptr<float>());

    for (int i = 0; i < detectionMat.rows; i++)
    {            
        if (detectionMat.at<float>(i, 2) >= 0.5)
        {
            FaceDetectionResult res;
            res.faceDetected = true;
            res.confidence = detectionMat.at<float>(i, 2);

            res.x1 = static_cast<int>(detectionMat.at<float>(i, 3) * frameWidth);
            res.y1 = static_cast<int>(detectionMat.at<float>(i, 4) * frameHeight);
            res.x2 = static_cast<int>(detectionMat.at<float>(i, 5) * frameWidth);
            res.y2 = static_cast<int>(detectionMat.at<float>(i, 6) * frameHeight);

            vec.push_back(res);
        }
#ifdef aDEBUG
        else
        {

            cout << detectionMat.at<float>(i, 2) << endl;

        }
#endif
    }                
    return vec;
}

And the original image which I am using is this.

While searching the cause behind this issue, one of my stack overflow friend asked me to crop the image and perform detection on it. After cropping image to dimension 481x576 (original was 962x576) opencv detected 18 faces. The same stack overflow friend given below expiation for why opencv detects faces after cropping and why not on full image

Below is the answer given on stackoverflow: (Link to Stackoverflow question for reference https://stackoverflow.com/questions/5...)

After a deep search, unfortunately I couldn't find a good explanation to this problem. The reason why I tried to crop image is that I assumed there can be a maximum detected face number limit. It is also not about occlusion.

I tried some image examples which includes more than 20(appx.) faces and the results were the same but when I cropped those images(decrease the number of faces), program was able to find the faces ...

(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
3

answered 2020-01-29 05:47:32 -0500

berak gravatar image

updated 2020-01-29 05:55:59 -0500

a quick experiment reveals, there is a maximum of 200 possible detections (in this model):

>>> net = cv2.dnn.readNet("opencv_face_detector_uint8.pb", "opencv_face_detector.pbtxt")
>>> net.setInput(np.ones((1,3,300,300),np.float32))
>>> out = net.forward()
>>> out.shape
(1, 1, 200, 7)

This is also not about the resolution

it probably is. the ratio of the crop size you give to blobFromImage() and your original image size determies the quality of the detection

edit flag offensive delete link more

Comments

Okay. So, can we say that, as maximum of 200 possible detection is the limit. So when we pass an image having so many faces (approx. 20) then relatively possible detection number go beyond 200 but as it's above the limit so we get only first 200 result in cv::Mat from line of code cv::Mat detection = net.forward("detection_out") and after that when we put confidentThreshold of 0.5 then within that 200 possibilities no faces are having that threshold.

Amogh gravatar imageAmogh ( 2020-01-29 06:01:36 -0500 )edit

So when we pass an image having so many faces (approx. 20) then relatively possible detection number go beyond 200

why do you think so ? it does not make any sense.

IF it detects 20 faces (prob > thresh), the other 180 (call it "slots" ?) will be below the threshold simply.

berak gravatar imageberak ( 2020-01-29 09:30:20 -0500 )edit

Then, I still have question why it cannot detect all faces even if I set threshold to .12 and when i crop image then it detects almost all.

Amogh gravatar imageAmogh ( 2020-01-29 10:49:06 -0500 )edit

because of the image resolution ?

(it was trained on 300x300 windows, so it gets worse, the larger your images are)

berak gravatar imageberak ( 2020-01-29 10:56:28 -0500 )edit

If you checked the code then, i m resizing the image to 300x300. Even when i cropped image then the resolution was 481x578 then 18 faces are detected

Amogh gravatar imageAmogh ( 2020-01-29 11:04:40 -0500 )edit

if i check which code ? (there's none here)

berak gravatar imageberak ( 2020-01-29 11:07:56 -0500 )edit

Sorry, actually it is posted on stackoverflow, link to SO question is alredy added in the question here.

Amogh gravatar imageAmogh ( 2020-01-29 11:16:45 -0500 )edit

yea, that's why cross posting is useless

berak gravatar imageberak ( 2020-01-29 11:42:21 -0500 )edit
1

Im sorry again. :(, I updated the question with code.

Amogh gravatar imageAmogh ( 2020-01-29 11:55:12 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-01-29 05:26:26 -0500

Seen: 331 times

Last updated: Jan 29