Face Landmarks and stabilization

asked 2019-02-22 09:38:08 -0500

Sartogersofortotropher gravatar image

updated 2019-02-23 02:43:47 -0500

Points on my landmarks are jitter.

How I can stabilize my face landmark?

Do I can use calcOpticalFlowPyrLK?

How I can use calcOpticalFlowPyrLK?

My code:

#include <iostream>
#include <cmath>

#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/objdetect/objdetect.hpp>

#include <opencv2/face.hpp>

using cv::Scalar;
using cv::Point;

int main(int argc, char** argv)
{
    cv::CascadeClassifier faceDetector("haarcascade_frontalface_alt2.xml");

    cv::Ptr<cv::face::Facemark>facemark = cv::face::FacemarkLBF::create();

    facemark->loadModel("lbfmodel.yaml");

    cv::VideoCapture vc(0);

    while (true)
    {
        cv::Mat frame, gray;

        vc.read(frame);

        cv::cvtColor(frame, gray, cv::COLOR_BGR2GRAY);
        //
        std::vector<cv::Rect> faces;

        faceDetector.detectMultiScale(gray, faces);

        std::vector< std::vector<cv::Point2f> > landmarks;
        //// Run landmark detector

        bool success = facemark->fit(frame, faces, landmarks);
        for (size_t i = 0; i < landmarks.size(); i++)
        {
            for (size_t j = 0; j < landmarks[i].size(); j++)
            {
                cv::circle(frame, cv::Point(landmarks[i][j].x, landmarks[i][j].y), 2, Scalar(255, 0, 0), 2);
            }
        }
        cv::imshow("1", frame);

        if ((char)cv::waitKey(20) == 27)
            break;

    }


    return 0;
}
edit retag flag offensive close merge delete

Comments

usually, it's the face detection, which is noisy. this can be overcome by using a Tracker, like CSRT or MOSSE instead of consecutively detecting faces.

berak gravatar imageberak ( 2019-02-22 15:11:22 -0500 )edit

@berak, UPD

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-23 02:00:07 -0500 )edit

UPD means ?

berak gravatar imageberak ( 2019-02-23 02:04:26 -0500 )edit

@berak, UPD - UPDate

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-23 02:06:41 -0500 )edit

ahh, ok, should have finished my coffee before reading ;)

how would sparse optical flow help here ? again, to my exp., it's the cascade detection, which introduces the noise.

berak gravatar imageberak ( 2019-02-23 02:13:55 -0500 )edit

@berak, ok, but I can't understand, how fix that?

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-23 02:15:13 -0500 )edit

@berak, is it hard?

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-23 02:38:01 -0500 )edit

try, if you get a more stable box from this, using CSRT or MOSSE

berak gravatar imageberak ( 2019-02-23 02:55:28 -0500 )edit

@berak, oh, I stabilize my cascade detection, but landmarks aren't stabilize in box. I can save prev box and if new box changed is small I use prev box. but points are jitters :(

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-23 03:04:33 -0500 )edit

@berak, ??

Sartogersofortotropher gravatar imageSartogersofortotropher ( 2019-02-24 03:01:53 -0500 )edit