Face detect inconsistent performance

asked 2015-10-03 14:48:31 -0600

slycheese
11 ●1 ●3

updated 2015-10-22 17:46:30 -0600

I'm using the cascade classifier with the front-face training data to detect faces in a still image (i.e. static jpg). I don't want to have to search the whole image for faces at once, so I send the cascade classifier an ROI from the full image. What I've found is that the classifier's ability to find the face in the image is dependent on the dimensions/position of the ROI I give it. Note, I only give it ROIs that include the face fully, and with lots of margin.

I'm using OpenCV 3.0.0.

I had modified the OpenCV sample program to demonstrate the behavior, but was then informed that that sample program was old and used an interface to the cascade classifier that is no longer valid. I have modified the new OpenCV sample program (obtained from: http://docs.opencv.org/master/db/d28/...) but that code can't find the face in my sample image at all (see below), which is the same image I have been using all along. I've also found that when I test some other images, the sample program seg faults.

Sample image: C:\fakepath\faceDetectTestImage.png

Sample image that causes seg fault: image description

Backtrace after seg fault:

Program received signal SIGSEGV, Segmentation fault.
0xb7816a42 in cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point_<int>, int, double&) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
(gdb) bt
#0  0xb7816a42 in cv::CascadeClassifierImpl::runAt(cv::Ptr<cv::FeatureEvaluator>&, cv::Point_<int>, int, double&) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
#1  0xb781d189 in cv::CascadeClassifierInvoker::operator()(cv::Range const&) const () from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
#2  0xb7ed12d7 in cv::parallel_for_(cv::Range const&, cv::ParallelLoopBody const&, double) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_core.so.3.0
#3  0xb781f068 in cv::CascadeClassifierImpl::detectMultiScaleNoGrouping(cv::_InputArray const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, std::vector<int, std::allocator<int> >&, std::vector<double, std::allocator<double> >&, double, cv::Size_<int>, cv::Size_<int>, bool) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
#4  0xb7828956 in cv::CascadeClassifierImpl::detectMultiScale(cv::_InputArray const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, std::vector<int, std::allocator<int> >&, std::vector<double, std::allocator<double> >&, double, int, int, cv::Size_<int>, cv::Size_<int>, bool) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
#5  0xb7812af1 in cv::CascadeClassifierImpl::detectMultiScale(cv::_InputArray const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, double, int, int, cv::Size_<int>, cv::Size_<int>) ()
   from /home/mss/src/opencv-3.0.0-beta/build/lib/libopencv_objdetect.so.3.0
#6  0xb78171a6 in cv::CascadeClassifier::detectMultiScale(cv::_InputArray const&---Type <return> to continue, or q <return> to quit---
, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > >&, double, int, int ...

(more)

edit retag flag offensive close merge delete

Comments

Couple of things:

"I've found is that the classifier's ability to find the face in the image is dependent on the dimensions/position of the ROI I give it" - dependent on the dimensions/position in what way? Because you later say that using other test images changes ROIs that don't work
"The face detector works properly in about 90% of the ROIs I give it" - that's a pretty decent result. In detection issues, expecting a 100% is just a dream, unfortunately

LorenaGdL ( 2015-10-04 03:05:52 -0600 )edit

Start by providing us some results of in which cases it does not work, or where the result becomes worse. Since the whole rescaling for the multi scale detection is based on rescaling the original image, you can see how this can influence the detection. Also lowering the scale step, and adding thus more scales to evaluate could improve it alot but will slow your algorithm down.

StevenPuttemans ( 2015-10-05 06:33:55 -0600 )edit

LorenaGdL:

What I meant is that if you take a still photo and run face detection on it using various ROIs (each of which includes the face in its rectangle), for some ROIs the face is found and for others it isn't. I can see no pattern with regard to which ROIs work and which don't. ROIs that work/don't work are coupled with the still photo used, i.e., if you switch and use a different still image, the set of ROIs that work change. When I say "dimensions/position" I mean the size and width of the ROI, and the position of it's top-left corner, and as I said, I can not see a pattern in with locations/sizes of ROI work and with don't.
I think that the filter is deterministic, isn't it? If that's the case, then for a ...

(more)

slycheese ( 2015-10-06 14:13:48 -0600 )edit

@slycheese: please, follow @StevenPuttemans suggestion and show us some examples (you can edit and update your question). Also, please provide minimum code/parameters to see what's going on

LorenaGdL ( 2015-10-06 14:28:49 -0600 )edit

@LorenaGdL, @StevenPuttemans: I modified the OpenCV example face-detect code to show the issue. Thanks again for your thoughts on this.

slycheese ( 2015-10-08 15:28:23 -0600 )edit

Where did you get that sample o_O? It is mixing the old C and new C++ API which is the worst possible thing you can do ... that will likely be the cause of the problem ...

StevenPuttemans ( 2015-10-13 02:37:53 -0600 )edit

@StevenPuttemans in fact, does OpenCV 3.0 even work only with the old C-api? I don't think so

LorenaGdL ( 2015-10-13 02:41:11 -0600 )edit

@LorenaGdL, some of the C++ functions still perform underlying calls to the old C-API but like you say, I would not use the C API withing 3.0, because it will screw up the referencing of Mat inner pointers.

StevenPuttemans ( 2015-10-13 03:23:00 -0600 )edit

@slycheese: you can find an updated tutorial here

LorenaGdL ( 2015-10-13 03:39:55 -0600 )edit

@StevenPuttemans I'm not sure what "sample o_O" means, but the code I posted is what's in "samples/cpp/facedetect.cpp" in the 3.0.0 codebase, with my noted modifications.

slycheese ( 2015-10-13 14:23:45 -0600 )edit

see more comments

answered 2015-10-22 18:56:18 -0600

sturkmen

6772 ●3 ●48 ●79 https://github.com/stu...

i tested the given image by adding the lines below. it seems @slycheese is right.

"classifier's ability to find the face in the image is dependent on the dimensions/position of the ROI I give it"

/*** Mod begin */
if(!inputName.empty())
{
    frame = cv::imread(inputName);

    for(int percent = 100; percent > 40; percent-- )
    {
        Rect newroi;
        newroi.width=(frame.cols*percent)/100;
        newroi.height=(frame.rows*percent)/100;
        newroi.x=(frame.cols-newroi.width)/2;
        newroi.y=(frame.rows-newroi.height)/2;

        Mat shrinked = frame(newroi).clone();
        detectAndDisplay( shrinked );
        waitKey();
    }
}
else
    /*** Mod end */

edit flag offensive delete link

Comments

Very interesting. I can reproduce the error too. Definitely, the CascadeClassifer needs a makeover

LorenaGdL ( 2015-10-23 02:34:56 -0600 )edit

Also reproduced with 2.4.12. However, detection results are much better - there are less non-detections, and not a single false detection

LorenaGdL ( 2015-10-23 02:48:21 -0600 )edit

Wow percentages and integer divisions :D auch!

StevenPuttemans ( 2015-10-23 02:50:10 -0600 )edit

Hmmm still I think what you guys are doing is dangerous. You are taking a smaller image or larger image, but you keep the same stepsize in the image pyramid. This will result in more or less scale levels being evaluated and THUS normally the detection result will change. If you want a good comparison you need to change the detection parameters with each resize to have the exact same scale pyramid before you can do a good comparison.

StevenPuttemans ( 2015-10-23 02:52:50 -0600 )edit

@StevenPuttemans didn't think about that. Anyway, 2.4.12 results are in my opinion within expectations (you will almost always have some non detections); however, it is true that for 3.0.0 the trained classifier does not work particularly well, probably due to all those changes between versions. I think new .xml files should be supplied if possible, or at least somewhere should be written "careful, may not worked as expected"

LorenaGdL ( 2015-10-23 03:06:38 -0600 )edit

Hmmm that might be true ...

StevenPuttemans ( 2015-10-23 03:42:24 -0600 )edit

@StevenPuttemans Can you offer any advice on how one should select the stepsize in the image pyramid given a particular resolution input image?

For reference, the reason I'm changing the search ROI is because once I find where the face is in the relatively large input image, on subsequent frames I want to only search in a smaller ROI where the face was previously found (since it's unlikely that it moved very far in one frame). This seems like something that would be a fairly common practice.

slycheese ( 2015-10-25 23:51:17 -0600 )edit

add a comment

Face detect inconsistent performance

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Face detect inconsistent performance edit

Comments

1 answer

Comments

Links

Question Tools

Stats

Related questions

Face detect inconsistent performance