1 | initial version |
I'm not well-versed in OpenCV, but I have a working gender classification program, and I am using exactly the same method as you. I still have it at around 75% accuracy, but it can all be improved afterthat. Like what you said, I believe it is returning the most matching face too, but that doesn't mean it can't be believed.
The thing about it is to have a more diverse set of face database, it'll be good to increase the number of faces you have for both genders, with different races, different emotions (or void of any). It'll be good to include a few faces that is unique, for example, faces of male that look feminine, or faces of female that looks boyish. The larger your database, the more diverse it is, I believe it'll return better results.
20 faces for each class is pretty small, many others recommend to have a few hundreds.
There are definitely many other ways to do it, or even better ways, but what you're doing is not wrong either, and there are people who reached up to over 95% of accuracy, so if you don't have a lot of time to learn about a total new algorithm or method, I think it'll be good to stick with Fisherfaces and just add on to your face database. It'll take a lot of time too, but a more diverse and large set of training data would help in any approach you're taking in gender classification I believe.