Ask Your Question
1

Face embedding calculation from Java

asked 2019-10-21 23:26:25 -0600

mzattera gravatar image

updated 2019-10-22 04:11:30 -0600

berak gravatar image

Hi, is there a way to calculate face embedding from the Java port of OpenCV?

I have browsed the JavaDoc but I cannot find it.

Thanks in advance.

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
2

answered 2019-10-22 01:02:02 -0600

berak gravatar image

you're lucky, here's a complete example:

import org.opencv.core.*;
import org.opencv.dnn.*;
import org.opencv.imgcodecs.*;
import org.opencv.imgproc.*;

public class FaceRecognition {
    public static Mat process(Net net, Mat img) {
        Mat inputBlob = 
            Dnn.blobFromImage(img, 1./255, new Size(96,96), new Scalar(0,0,0), true, false);
        net.setInput(inputBlob);
        return net.forward().clone();
    }
    public static void main(String[] args) {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        Net net = Dnn.readNetFromTorch("openface.nn4.small2.v1.t7");
        Mat feature1 = process(net, Imgcodecs.imread("Abdullah_Gul_0004.jpg")); 
        Mat feature2 = process(net, Imgcodecs.imread("Abdullah_Gul_0007.jpg")); 
        double dist  = Core.norm(feature1,  feature2);
        if (dist < 0.6)
            System.out.println("SAME !");
    }
}
edit flag offensive delete link more

Comments

Once you got the embedding vectors(the difficult part) - just use euclidean distance(easy part).

holger gravatar imageholger ( 2019-10-22 02:53:06 -0600 )edit

Hi. I keep having strange results. I am running the above code verbatim (no changes).I am using OpenCV 4.1.1 I downloaded the Java libraries from Sourceforge and I am running under Windows 10 64 bit.

I downloaded the model from here: https://storage.cmusatyalab.org/openface-models/nn4.small2.v1.t7

I downloaded test images from here: https://github.com/muxspace/facial_expressions/tree/master/images

When I run the code, it gives me a distance of 0.9325596751685123 between the two faces. Clerly the embedding is not working :(

mzattera gravatar imagemzattera ( 2019-10-23 13:15:23 -0600 )edit
1

I think the reason is the face should be aligned first. And I fear OpenCV has no Java libraries for that. All the examples I found in Python use dlib for which I could not find a Java port.

mzattera gravatar imagemzattera ( 2019-10-30 07:32:01 -0600 )edit
1

I think the reason is the face should be aligned first. And I fear OpenCV has no Java libraries for that. All the examples I found in Python use dlib for which I could not find a Java port.

mzattera gravatar imagemzattera ( 2019-10-30 07:32:02 -0600 )edit
1
2

answered 2019-10-22 09:45:50 -0600

mzattera gravatar image

updated 2019-10-22 10:02:10 -0600

berak gravatar image

Thanks a lot for the quick reply! However, let me add a note fro beginners like me :)

When I ran the above code, I got the following error:

 cannot open <openface.nn4.small2.v1.t7>

This because, after installing OpenCV for Java as explained here, you need to download the model separately from here . Then, you need to pass the full file name for the downloaded model in:

 Dnn.readNetFromTorch( [fullFileNameHere] );
edit flag offensive delete link more

Comments

yea indeed, your answer here is nessecary to complete it, so thank you ;)

berak gravatar imageberak ( 2019-10-22 09:57:18 -0600 )edit

But you also need to download "Abdullah_Gul_0004.jpg" - so its common sense :-) Anyways - not my thread - and excuse my "micro trolling"

holger gravatar imageholger ( 2019-10-22 12:35:14 -0600 )edit

^^ good point, actually, so:

  • for now, you HAVE TO use the presupplied openface.nn4.small2.v1.t7 model
  • but YOU'RE FREE to choose your own images.
berak gravatar imageberak ( 2019-10-22 12:40:20 -0600 )edit
1

It seems I get better results (but I am checking only with 2 people having 5 pictures each) with:

Dnn.blobFromImage(img, 1./255, new Size(96,96), new Scalar(0,0,0), false, false);

So if I put swapRB = false...is it only my dataset or a bug in the above code?

mzattera gravatar imagemzattera ( 2019-10-23 04:39:42 -0600 )edit

uhhhmm -- maybe you found something !

code was adapted from the js sample but it looks like the js code is working on RGBA, not BGR

berak gravatar imageberak ( 2019-10-23 04:50:09 -0600 )edit

....still, cannot fix the returned embedding. The above code is not able to identify faces as belonging to the same person.

mzattera gravatar imagemzattera ( 2019-10-30 05:37:00 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2019-10-21 23:26:25 -0600

Seen: 498 times

Last updated: Oct 22 '19