Eye detection is "jumping". [closed]
I am working on a system that tracks the eye and log data like eye blinks, rate, etc. But I have a problem that the cv::Rect, returned after eye detection (haar detection), is jumping around so that the iris of the person appear on different positions in the matrix. in a 640x480 web cam frame, where the face is about 200px height the cv::Rect of the eye can jum about 40px each time I do the tracking. It jumps so far out that the eye sometimes is outside the rectangle representing the eye!
I am running this detection about 2 times per second, and in between I trace the eye by template mathcing. The template matching works great but sometimes the template slides of and after that keep track on the wrong position of the face. For example during a fast movement of the face the template can get stuck on the nose, or mouth.
I've tried 3 of the four eye classsifiers, (_eye, _mcs, _lefteye_2split). These three, according to the book "Master openCV..." is the best. But I cant se any big difference (more than the _mcs classifier is much slower)
It doesnt matter which one I use, So i was thinking that I might do something wrong!
Someone that is running eye detection multiple times with a good result?
My detector: eyeCascade.detectMultiScale(webcamFrame(ROI), detEye, 1.1, 3, CV_HAAR_FIND_BIGGEST_OBJECT|CV_HAAR_DO_CANNY_PRUNING|CV_HAAR_SCALE_IMAGE, cv::Size(20,20
ROI = centerpoint of last eye position, double width, double height. (4x sixe of cv::Rect( eye position))
edit: There are different classifiers for left and right eye. I force the system to find only one eye. (persons right, on the screen the left one.) Which classifier is the right one?
kalman filter ?
No filtering of the position yet. Do you think that is necessary?, I mean... The cv::Rect output from detection algorithm misses the eye abou 1/3 of all the times. Half the pupil is inside the rectangle. It feels wrong. Head is still, image is good.
"Do you think that is necessary ?"
ah, no idea, just mindstorming
Why using haar classifiers? Did you try flandmarks for example in order to get the position of the eyes and another landmarks? And this paper "Accurate eye centre localisation by means of gradients" has a c++ implementation for pupil tracking
I took a look at the flandmarks stuff, after your recommendation, and it sounds really good. Is it possible to use it in real - time you think? Haar vs flandmarks -computation time? Thanks! I will try flandmarks
Haar vs flandmarks computation time i don't know. But if you look at the videos, you can see the fps of flandmarks with different resolutions.
i cant get the flandmarks to work. unresolved external symbol during FLANDMARK_init(); :(
you need the code from the cpp/libflandmark folder in your project:
and later, when you run it, it needs to find flandmark_model.dat ( from the data folder )
There is no cpp folder, but I copied the files from libflandmark into my project. stdint.h isnt there, but a file called msvc-compat.h. I copied 5 files. I include the dlandmark_detecter.h into my main file. I am able to "FLANDMARK * model;", but when I try to do "model = flandmark_init("file"); I get the unresolved external symbol. Where shall I start to look for the error? Thanks for helping me!
oh, right, i got the old sourcefourge version ..
still investigating!