Ask Your Question
1

Online Learning and Confidence with cv::FaceRecognizer

asked 2012-08-24 04:58:55 -0500

ranjanritesh gravatar image

updated 2012-08-24 05:03:53 -0500

platform: Ubuntu 12.04 LTS 64bit

OpenCV version: 2.4.2

I am trying to make a program that captures face (say user 1) using haarcascade.I then run the crop the face are of image and add it to a vector<mat>.I then add diff faces(say user 2) using the same method to the same vector. But I add different labels (say 0 and 1) for the 2 different cases. After some kind of keyboard interrupt,I train the FaceRecognizer using train() with both the vector (Mat and int). From the next frame onwards I take the face area and try to predict the label.

The code is compiling and running fine.But the outputs are a little irritating. It always outputs the same label( on more inspection I found, the label that I "push_back" for the very first frame) and the confidence is always 0.

What I am basically trying to achieve is a kind of online learning FaceRecognizer. But Since the predicted labels are not correct,I assume I have done something wrong. Is online learning even possible with PCA/LDA/LBPH?? I have tried using the same model as well as saving and opening with another model. Below is my code.Am I doing something wrong??Any help will be much appreciated!!! Thanks

    using namespace cv;
    using namespace std;
    vector< Rect_<int> > faces;

    vector<cv::Mat> learnt_face;
    vector<int> learnt_label;

    bool pred = false;
    bool pos_ex = false;

    int main(int argc,char* argv[])
    {
        cv::CascadeClassifier haar_cascade;

        Ptr<cv::FaceRecognizer> model = cv::createEigenFaceRecognizer(0,140.0);
        Ptr<cv::FaceRecognizer> model0 = cv::createEigenFaceRecognizer(0,140.0);
        string fn_haar = string("haarcascade_frontalface_alt.xml");
        haar_cascade.load(fn_haar);

        VideoCapture cap(0);

        Mat img,gray_img,crop_face,crop_face_res;
        for(;;)
        {
            cap>>img;
            cv::cvtColor(img,gray_img,CV_RGB2GRAY);

            haar_cascade.detectMultiScale(gray_img, faces); 

                Rect face_i = faces[0];                 //Take only one face at a time * For debugging Purposes

                //Crop and Resize
                crop_face = gray_img(face_i);
                cv::resize(crop_face, crop_face_res, Size(100,100), 1.0, 1.0, INTER_CUBIC);

                if((!crop_face.empty()) && pred == false && pos_ex == true ) //If cropped,not predicting and learning positive images
                {
                    cout<<"Learning"<<endl;

                    learnt_face.push_back(crop_face_res);
                    learnt_label.push_back(0);

                    if(learnt_face.size() >= 100)
                    {
                        learnt_face.erase( learnt_face.begin());
                        learnt_label.erase( learnt_label.begin());
                    }
                    rectangle(img, face_i, CV_RGB(0, 255,0), 1);    //Green Faces label 0.

                    model->train(learnt_face,learnt_label);

                }
                //If cropped,not predicting and learning negetive images
                else if((!crop_face.empty()) && pred == false && pos_ex == false )
                {
                    cout<<"Learning"<<endl;
                    learnt_face.push_back(crop_face_res);
                    learnt_label.push_back(1);

                    if(learnt_face.size() >= 100)
                    {
                        learnt_face.erase( learnt_face.begin());
                        learnt_label.erase( learnt_label.begin());
                    }
                    rectangle(img, face_i, CV_RGB(255, 0,0), 1); //Red faces label 1.

                    model->train(learnt_face,learnt_label);

                }
                //If cropped and predicting 
                else if((!crop_face.empty()) && pred == true)
                {

                        //model->save("model.xml");
                        //model0->load("model.xml");

                    cout<<"Predicting"<<endl;
                    int prediction = -1;
                    double predicted_confidence = 0.0;
                    cout<<model->getDouble("threshold")<<endl;
                    model->predict(crop_face_res,prediction,predicted_confidence);
                    rectangle(img, face_i, CV_RGB(0, 0,255), 1);

                    string box_text = format("Prediction = %d  Confidence = %f", prediction,predicted_confidence);

                    int pos_x = std::max(face_i.tl().x - 10, 0);
                    int pos_y ...
(more)
edit retag flag offensive close merge delete

Comments

I have even tried changing the thresholds ... anything higher than 0 gives the same problem and 0,as expected gives -1 label

ranjanritesh gravatar imageranjanritesh ( 2012-08-24 05:17:54 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
3

answered 2012-10-07 13:20:28 -0500

updated 2012-10-07 22:40:39 -0500

Neither a classic Principal Component Analysis (PCA) nor a classic Linear Discriminant Analysis (LDA) can be used for online learning. You can easily verify this by having a look at the mathematical details outlined at:

Local Binary Patterns Histogram (LBPH) instead don't build a model explicitly, so this FaceRecognizer can be updated with cv::FaceRecognizer::update and used for online learning as such. However, there are also incremental versions of a PCA and LDA you could try implementing. See:

  • Haitao Zhao, Pong Chi Yuen and James T. Kwok. "Incremental Principal Component Analysis and its Application for Face Recognition" in IEEE Transactions on Systems, Man and Cybernetics (Part B), 36(4):873-886, Aug 2006. [PDF Online Available here]

If you want to experiment with the incremental versions real quick, you could try to start with an available Python implementation and interface with the OpenCV Python binding. An implementation of the Incremental PCA is given at:

I would strongly suggest validating it on your datasets, to see what the performance of an Incremental PCA really is. If there's enough interest I can write a small script to evaluate it. I don't want to advertise my own answers, but probably this post is helpful in order to analyze a classifiers performance:

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

2 followers

Stats

Asked: 2012-08-24 04:58:55 -0500

Seen: 1,513 times

Last updated: Oct 07 '12