Head pose estimation with OpenCV java
I am trying to do head pose estimation with OpenCV java. I am trying to follow http://free-d.org/3d-head-tracking-in... tutorial
now for far i can track the face but how can i use the algorithm once the face is tracked
here are my codes
main:
public class App {
private JFrame frame;
private JLabel label;
private void mainGUI(){
frame= new JFrame("The Camera");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setSize(300,300);
label=new JLabel();
frame.add(label);
frame.setVisible(true);
}
/**** Run main Loop for face detection and all other good stuff ****/
private void runLoop(){
String frontFacePath= "/opencv/data/haarcascades/haarcascade_frontalface_alt.xml";
CascadeClassifier faceCascade = new CascadeClassifier(frontFacePath);
Mat webCamImg = new Mat();
Image tmpImg;
VideoCapture capture = new VideoCapture(0);
double fps = capture.get(Videoio.CAP_PROP_FPS);
System.out.println(fps);
capture.set(Videoio.CAP_PROP_FRAME_WIDTH,300);
capture.set(Videoio.CAP_PROP_FRAME_HEIGHT,300);
ConvertImage convertImage = new ConvertImage();
if (capture.isOpened()){
while (true){
capture.read(webCamImg);
MatOfRect faceRect = new MatOfRect();
MatOfRect eyeRect = new MatOfRect();
MatOfRect faceProfile = new MatOfRect();
faceCascade.detectMultiScale(webCamImg,faceRect);
eyeCascade.detectMultiScale(webCamImg,eyeRect);
profileCascade.detectMultiScale(webCamImg,faceProfile);
for (Rect rect:faceRect.toArray()){
Imgproc.putText(webCamImg,"face",
new Point(rect.x,rect.y-5),1,1,
new Scalar(0,0,225));
Imgproc camShow = new Imgproc();
camShow.rectangle(webCamImg,
new Point(rect.x,rect.y),
new Point(rect.x+rect.width,rect.y+rect.height),
new Scalar(0,100,200),5);
if (capture.grab()){
System.out.println("Frame Grabbed ");
/*** Calll the Head Pose estimation method **/
HeadPoseEstimationMain hpE = new HeadPoseEstimationMain();
//hpE.HPE(webCamImg);
}
System.out.println("Face");
}
if (!webCamImg.empty()){
tmpImg= convertImage.convertMatToImg(webCamImg);
ImageIcon imageIcon = new ImageIcon(tmpImg,"Captured");
label.setIcon(imageIcon);
frame.pack();
}
else {
System.out.println("no frame");
break;
}
}
}
else {
System.out.println("nothing ");
}
}
/**---------------------------- The main --------------------------**/
public static void main(String[] args) {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
App app = new App();
app.mainGUI();
app.runLoop();
}
}
The Heap pose estimation class
public class HeadPoseEstimationMain {
public void HPE(Mat matOfImg){
int MaxCount= 10;
TermCriteria termCriteria = new TermCriteria(TermCriteria.EPS | TermCriteria.COUNT,20,0.3);
List<MatOfPoint2f> matOfPointList = new ArrayList<MatOfPoint2f>();
Size subPixWinSize = new Size(10,10);
Size winSize = new Size (21,21);
Imgproc.goodFeaturesToTrack(matOfImg, (MatOfPoint) matOfPointList,MaxCount,0.01,10, null,3,true,0.04);
Imgproc.cornerSubPix(matOfImg, (MatOfPoint2f) matOfPointList,subPixWinSize,new Size(-1,-1),termCriteria);
}
}
Please help me out
imho, your tutorial there is quite bad. it does not show the 2d ->3d mapping (solvePnP), you probably should use some real facial landmarks, and a real 3d head model.
maybe this or that
Thanks for the Hint. How can i Add those model points with Java ?
is it kind of List<point3> point3List = new ArrayList<>();
yea, something like that.
but the gist is: you need some 3d coords corresponding to some known 2d landmarks, then you go and find the landmarks in your image, and use solvePnP with the new 2d positions, and the original 3d points, to get a (3d) transformation matrix - the "pose"..
the corner-tracking / optflow thing in your original post won't lead you anywhere, simply said.