OpenCV android projectPoints - Assertion failed

asked 2016-11-02 16:09:37 -0600

Hello, I'm trying to get a head pose in real time using OpenCV on android according to this tutorial: http://www.learnopencv.com/head-pose-...

This is the source code of the program he used in the video: https://github.com/spmallick/dlib/blo...

These are my parameters initialization:

public MatOfPoint3f get_3d_model_points()
{
    List<Point3> objectPointsList    = new ArrayList<Point3>(6);
    objectPointsList.add(new Point3(0.0f, 0.0f, 0.0f));
    objectPointsList.add(new Point3(0.0f, -330.0f, -65.0f));
    objectPointsList.add(new Point3(-225.0f, 170.0f, -135.0f));
    objectPointsList.add(new Point3(225.0f, 170.0f, -135.0f));
    objectPointsList.add(new Point3(-150.0f, -150.0f, -125.0f));
    objectPointsList.add(new Point3(150.0f, -150.0f, -125.0f));

    MatOfPoint3f modelPoints = new MatOfPoint3f();
    modelPoints.fromList(objectPointsList);

    return modelPoints;

}

public MatOfPoint2f get_2d_image_points(ArrayList<Point> d)
{
    List<org.opencv.core.Point> imagePointsList      = new ArrayList<org.opencv.core.Point>(6);
    imagePointsList.add(new org.opencv.core.Point(d.get(30).x, d.get(30).y));
    imagePointsList.add(new org.opencv.core.Point(d.get(8).x, d.get(8).y));
    imagePointsList.add(new org.opencv.core.Point(d.get(36).x, d.get(36).y));
    imagePointsList.add(new org.opencv.core.Point(d.get(45).x, d.get(45).y));
    imagePointsList.add(new org.opencv.core.Point(d.get(48).x, d.get(48).y));
    imagePointsList.add(new org.opencv.core.Point(d.get(54).x, d.get(54).y));

    MatOfPoint2f modelPoints = new MatOfPoint2f();
    modelPoints.fromList(imagePointsList);
    return modelPoints;
}

Mat get_camera_matrix(float focal_length, Point center)
{
    Mat camera_matrix =  Mat.eye(3, 3, CvType.CV_32F);

    camera_matrix.put(0,0,focal_length);
    camera_matrix.put(0,1,0.0);
    camera_matrix.put(0,2,center.x);

    camera_matrix.put(1,0,0.0);
    camera_matrix.put(1,1,focal_length);
    camera_matrix.put(1,2,center.y);

    camera_matrix.put(2,0,0.0);
    camera_matrix.put(2,1,0.0);
    camera_matrix.put(2,2,1.0);
    return camera_matrix;
}

This is my code:

MatOfPoint3f model_points = get_3d_model_points();
MatOfPoint2f image_points  = get_2d_image_points(landmarks);
int focal_length = mCroppedBitmap.getHeight();

Mat camera_matrix = get_camera_matrix(focal_length, new Point(mCroppedBitmap.getWidth()/2, mCroppedBitmap.getHeight()/2));
MatOfDouble dist_coeffs = new MatOfDouble(Mat.zeros(4,1,CvType.CV_64FC1));

Mat rvec = new Mat();
Mat tvec = new Mat();

Calib3d.solvePnP(model_points, image_points, camera_matrix, new MatOfDouble(), rvec, tvec);

List<Point3> objectPointsList    = new ArrayList<Point3>(1);
objectPointsList.add(new Point3(0,0,1000.0));
MatOfPoint3f nose = new MatOfPoint3f();
nose.fromList(objectPointsList);

MatOfPoint2f nose2 = new MatOfPoint2f();

Calib3d.projectPoints(nose,rvec,tvec,camera_matrix,new MatOfDouble(),nose2);

This is the error i get:

OpenCV Error: Assertion failed (_tvec.total() * _tvec.channels() == 3 && (_tvec.depth() == CV_32F || _tvec.depth() == CV_64F)) in void    cv::fisheye::projectPoints(cv::InputArray, cv::OutputArray, cv::InputArray,    cv::InputArray, cv::InputArray, cv::InputArray, double, cv::OutputArray), file /home/maksim/workspace/android-pack/opencv/modules/calib3d/src/fisheye.cpp, line 77
11-02 22:50:28.403 26972-27078/com.tzutalin.dlibtest E/org.opencv.calib3d: calib3d::projectPoints_11() caught cv::Exception: /home/maksim/workspace/android-pack/opencv/modules/calib3d/src/fisheye.cpp:77: error: (-215) _tvec.total() * _tvec.channels() == 3 && (_tvec.depth() == CV_32F || _tvec.depth() == CV_64F) in ...
(more)
edit retag flag offensive close merge delete

Comments

hmmm, cv::fisheye::projectPoints <-- i'm pretty sure , you did not want to call this , reminds me of a recent bug

opencv version ? did you build the sdk locally (you couuld look into the generated src), or is it a prebuild one ?

berak gravatar imageberak ( 2016-11-03 02:28:28 -0600 )edit
1

I'm using OpenCV version 3.1 which i got from here: http://opencv.org/downloads.html

I just downloaded it and added the java folder in the sdk as a module.. How can i solve my problem?

Thanks

Liron Taub gravatar imageLiron Taub ( 2016-11-03 03:47:35 -0600 )edit

no java here atm. , so i can't test, but this pr might have fixed it

unfortunately, to try, you would need to build from latest github src (master branch) locally - which again would require a c++ compiler, cmake, and ant tools.

berak gravatar imageberak ( 2016-11-03 03:56:26 -0600 )edit
  1. Can i make native java function that calls projectPoints in the correct way by myself?
  2. Is there any guide on how i can compile it locally?
  3. Do you suggest me to try and implement projectPoints() by myself in java language?

Thank you very much

Liron Taub gravatar imageLiron Taub ( 2016-11-03 04:02:56 -0600 )edit
  1. not really, it's a fault in the java wrapper (wrong c++ code gets called)
  2. see link in comment above
  3. maybe. that's what i'm thinking about now, too. something like Rodrigues() (to get the transformation Mat from rvec & tvec), a matrix multiply with the camera mat, then multiply all the 3d points with that, but the details ... hmmm
berak gravatar imageberak ( 2016-11-03 04:07:51 -0600 )edit

Where can i see how projectPoints implemented? or at least the idea behind it in details.. Search in the OpenCV docs but found nothing :/ Btw do you think this bug exist in OpenCV 2 too?

Liron Taub gravatar imageLiron Taub ( 2016-11-03 04:11:36 -0600 )edit
  1. https://github.com/opencv/opencv/blob... (but no idea, what all those other weird cases there do). also look at http://docs.opencv.org/master/d9/d0c/...
  2. well the problem arose with the addition of the fisheye code, so maybe finding an earlier version helps
berak gravatar imageberak ( 2016-11-03 04:20:25 -0600 )edit
1

I'm building the latest opencv now from the guide you sent me.. i will update you if it will work..

Liron Taub gravatar imageLiron Taub ( 2016-11-03 04:50:25 -0600 )edit

Well, i compiled everything like in the guide you sent me but now i'm confused i got these files: https://s21.postimg.org/cpveijn2v/res...

I have no idea how to proceed now...

Liron Taub gravatar imageLiron Taub ( 2016-11-03 06:37:54 -0600 )edit

after you ran cmake && make && make install,

you should have an updated opencv-310.jar (and so) in your install folder (which is probably: /usr/local/share/Opencv )

berak gravatar imageberak ( 2016-11-03 07:21:50 -0600 )edit