offtopic: facenet model phenomenon [closed]

edit

could you add, how you calculate the "similarity score" ?

Yes - of course - i made sure the detection itself works correctly (the code you provided) and took this as a template https://github.com/opencv/opencv/blob...

My code (to big to paste it full)

//method to create embedding base on an extracted face
Mat createEmbedding(Mat face) {
    Mat blob = blobFromImage(face, 1.0 / 255, Size(96, 96), Scalar(0,0,0,0), true, false);
    identityNet.setInput(blob);
    Mat result = identityNet.forward().clone();
    return result;
}

//compute score using .dot - using it on the anchor itself give me similarity of 1.0 as expected
double score = anchorIdentity.dot(embedding);
cout << "score for anchor " << score << endl;

I could upload the full code (80 lines) to github

Hmm you give me an idea - maybe i shouldnt do it all together but extract the faces first and write them to disk and take this as input to further analyse. Will do this.

• time to read the paper, again: arXiv:1503.03832v3 (chapter 3, figure 3)
• here is, how triplet_loss would be calculated during the training phase

Ok if i understand you correctly(please comment):

• Dont copy some code from the js example
• Don't use the dot product
• Read about the euclidean distance calculation again and implement it correctly

Did i got you right? I hope some because then i can "fix" this. Thank you, Holger

no, sorry, the dot product is ok (since the embeddings are L2 normalized, i forgot that)

ok then - i am not complaining about facenet at all. The positives are valid and ok (score > 0.75). I am just wondering about the negatives and their probability. From a human perspective its really stupid.

How can lena have higher prob than a picture of arnold himself. If a human would take a look he could clearly tell that lena (beautiful woman) is obviously not more similar than a picture of an older arnold.

Anyway - maybe this leads to nothing and should just accept the fact that the models contain some "magiic" it learned from pixels and is not a human.

Ok then - converting it to a 3 channel black and white image produces more reasonable result for my little dataset

cvtColor(f,f,CV_RGB2GRAY); 
cvtColor(f,f,CV_GRAY2RGB);

Interesting. My theory is that this way i force the net to pay more attention on face landmarks. Only a theory - i need to verify on a bigger dataset (LFW dataset). But maybe i just f* up something this way, i will need to measure.

that the models contain some "magiic" it learned from pixels and is not a human.

hehe, right. it's also a knpwn fact, that you can fool cnn's e.g. with adversarial noise

well i guess a neuronal network just "sees" things just differently and is maybe right in its own world. Anyway - interesting discussion but will close it.

Thank you for your input! Greetings, Holger