kalman filter ?

No filtering of the position yet. Do you think that is necessary?, I mean... The cv::Rect output from detection algorithm misses the eye abou 1/3 of all the times. Half the pupil is inside the rectangle. It feels wrong. Head is still, image is good.

"Do you think that is necessary ?"

ah, no idea, just mindstorming

Why using haar classifiers? Did you try flandmarks for example in order to get the position of the eyes and another landmarks? And this paper "Accurate eye centre localisation by means of gradients" has a c++ implementation for pupil tracking

I took a look at the flandmarks stuff, after your recommendation, and it sounds really good. Is it possible to use it in real - time you think? Haar vs flandmarks -computation time? Thanks! I will try flandmarks

Haar vs flandmarks computation time i don't know. But if you look at the videos, you can see the fps of flandmarks with different resolutions.

i cant get the flandmarks to work. unresolved external symbol during FLANDMARK_init(); :(

you need the code from the cpp/libflandmark folder in your project:

flandmark_detector.cpp
flandmark_detector.h
liblbp.cpp
liblbp.h
stdint.h

and later, when you run it, it needs to find flandmark_model.dat ( from the data folder )

There is no cpp folder, but I copied the files from libflandmark into my project. stdint.h isnt there, but a file called msvc-compat.h. I copied 5 files. I include the dlandmark_detecter.h into my main file. I am able to "FLANDMARK * model;", but when I try to do "model = flandmark_init("file"); I get the unresolved external symbol. Where shall I start to look for the error? Thanks for helping me!

oh, right, i got the old sourcefourge version ..

still investigating!

Error LNK2001, and Error LNK1120.

This is the only thing I do:

include "stdafx.h"

include "flandmark_detector.h"

int _tmain(int argc, _TCHAR* argv[]) { FLANDMARK_Model * model = flandmark_init("flandmark_model.dat"); return 0; }

ahh those pesky tchars ..., might be a widechar / unicode / ascii thing then

afaik, if you use _tmain() it assumes all strings being tchars, so the function sig does not match (thus linker error)

try an empty project for now,

include "flandmark_detector.h"

int main(int argc, char * argv[]) { FLANDMARK_Model * model = flandmark_init("flandmark_model.dat"); return 0; }

I created an empty project. copied the files mentioned before. And used the code you wrote. I also included the propertysheey I use so that I can use opencv. But the same error. could it be the propertysheet? Just to be clear. I dont have to run this cmake-thing??!!

" I dont have to run this cmake-thing??!!"

not really. you'll have to add the opencv related includes/libs manually then

This is strange... Too bad there isn't much information available on the internet about flandmark. Thanks for your help!

at least i could reproduce it ;)

haven't found out what it is, but it is related to the propertysheet.

try to skip it as well, add the opencv stuff manually ( easier, if you edit the vcproj from scite or notepad )

By adding it manually, should I create an empty propertysheet and add the opencvstuff? or do you mean by copy all the files into the project folder?

Could it be because I use the debug-files from opencv? (with a 'd' at the end of each filename.

meant creating an empty project, and adding the "additional includes" in the c++ section / libs manually

unfortunately, i only got vs2008 here, so no real idea, how that looks on your side.

not likely. it's something in your project settings

I tried it again. Empty project. In the configuration manager I change to x64, (and copy from x86). I add the 5 files from the folder libflandmark. I add in C/C++ additional directories, opencv/build/include, -opencv & -opencv2. In the linker I add opencv/build/x64/vc10/lib/ and in linker-input I add all opencv_.....d.lib files. (debug files). Thats the only thing. Opencv works, And I still got a linker error from flandmarks... :(

I asked on stackoverflow.com. A guy there found that I didn't include the files in my project (add->existing files). WHen I did that everything works great. Thanks for helping me!

About the performance: With one (perfect image) when I measure the time for only the detect method, i get 0 ms. On my computer i thing I have a resolution of 15ms. So 1 detect (without the Haar-part) takes less than 15 ms. which is really really good!

Thank you everyone.

waa, what a story, nice that you got it solved ;)

so, does it give less jitter than the original (eye-cascade) idea ?

I have not tried it yet on webcam. But i tried about 10 photos. And the result were perfect!

ah, good. i asked, because i was trying asmlib for something similar before, and results on static images looked nice, too, but images from a webcam had the same jitter problems like your original cascade solution.

thanks for the info ;)

I will continue the work within a couple of days, and I can write a comment here about the result, after trying it with the webcam.