eye blinks detection from video frames

Hi

I want to detect blinks from video frames that recorded using eye glasses like the image in the attachment .

What is the best way to do this ?

Thanks

1.JPG

edit retag close merge delete

I'd try a circle fitting on the pupil, which seems to be clear enough..

( 2015-09-28 07:17:36 -0600 )edit

Sort by ยป oldest newest most voted

Just a quick try.. RESULT

When in a frame the circle cannot be detected then the eye is blinking.

What I've done in this test:

1. Use GaussianBlur to smooth the image
2. Threshold the image to keep only the darkest part, with THRESH_BINARY you can define a range of grey value to use as filter level
3. Use HoughCircles to fit circle in the image. Since the pupil is the only circle-like part in your image you can define a min/max radius value and get a unique circle
4. Draw the result
more

How this circle detects the eye ?

I'm a beginner in opencv , can you explain more ,

Thanks

( 2015-09-28 09:05:26 -0600 )edit

Edited the answer..in the documentation you will find easy examples on how to use those functions, added links too ;-)

( 2015-09-28 09:39:40 -0600 )edit

I have come up with this code , but I think it is not accurate , It can't track the eye when it moves in video

while(1)
{
Mat frame,grey,edge,draw,src_gray,dst;
cap >> frame;
cvtColor( frame, grey, CV_BGR2GRAY );
GaussianBlur( grey, src_gray, Size(9, 9), 2, 2 );
threshold(src_gray,dst, 0, 255, THRESH_BINARY);
SimpleBlobDetector detector;
std::vector<KeyPoint> keypoints;
detector.detect(  dst, keypoints);
Mat im_with_keypoints;
drawKeypoints( grey, keypoints, im_with_keypoints, Scalar(0,0,255), DrawMatchesFlags::DRAW_RICH_KEYPOINTS )

( 2015-09-28 10:33:58 -0600 )edit

Can you post the video or the frame where you can't track it? I think you're blurring the image too much, Size(3,3) will be fine..If you are missing some part of the pupil (because of the led lights reflected i suppose) you can try to erode and then dilate the image after thresholding, or adjust the radius range..that should fix the problem since the pupil is always clearly visible with a close-up image like that

( 2015-09-28 16:41:00 -0600 )edit

This is the video that I'm trying

video

( 2015-09-29 04:44:32 -0600 )edit

I'll try a simple approach, based on this paper [1]:

1. The LBP histogram describing the open eyes is constructed
2. Eye blinks are detected as sharp peaks of the dissimilarity between the template and the histogram of the current frame

[1] Malik, K., & Smolka, B. (2014, April). Eye blink detection using Local Binary Patterns. In Multimedia Computing and Systems (ICMCS), 2014 International Conference on (pp. 385-390). IEEE.

In order to build the LBP histogram of the eyes, you can use these other posts:

• Open eyes / closed eyes in Opencv

• how can I align Face Images

• Facial feature detection

• Facial features detection using haarcascade

• Are there any new ways to improve eye detection?

more