The challenge
I'm trying to replicate the experience from this campaign, but in realtime:
- http://www.gettyendlesspossibilities.com/
What I have now
- I'm able to detect faces and (somewhat naively) mark the regions I want: right and left eyes, cheeks, forehead, mouth, nose and so on.
- I run the same algorithm against a database of face images (5k images from Getty Images) and store image slices for each region.
- I'm just converting to gray and storing some metadata for each face slice. I suspect I could get better matches applying some kind of filter.
- I made a web app using Django Rest Framework that captures a snapshot using the web cam, locate your face and try to match each slice against slices of the same part of the face in the database.
The current problem
The matching is not very good. Sometimes, the best match for a face slice is a false positive from the face recognition algorithm (I got a low percentage of false positives and I'm curating the database by hand). I've tried the following algorithms in order to find the best match for each face feature from skimage.measure:
- compare_ssim
- compare_psnr
- compare_mse
- compare_nrmse
Best results are from Structural Similarity, but still far from ideal. Bellow are an example of the app returning a false positive as the best match for my mouth:
As you may have guessed I don't have a clue about what I'm doing - unable to figure out why it is not picking from hundred of mouths in the database.