haarcascade_mcs_lefteye.xml - for which eye?
I am going to use separate haar cascades to detect left and right eyes. My question is: is haarcascade_mcs_lefteye.xml applicable to the left eye of the person on the photo or to the eye at the left from the observer’s point of view? I cannot decide on my own because both haarcascade_mcs_lefteye.xml and haarcascade_mcs_righteye.xml detect both eyes on the photos that I have tried.
I use both for both cases then combine the detections. It is best do take this approach since in the end you will gather more detections and less misses! :)
What I do is add all detections into a single vector of rectangles. I then wrote a function called fuse_detections which takes a detection and checks if any of the other detections share more than 50% overlap. In that case, I merge both detections and create a bounding box containing the outer borders. Remove the two merged from the list and add the combined one. Keep going until no matches are found anymore. Then classify them as eyes. In order to know right and left, just analyze the coordinates of the corners. I used normal eye, right eye and left eye model. Running that on a detected face area takes almost no time and it gives a chance that one model missed but the other succeeded in detecting.