OpenCV Android. Error (assertion failed) in projectPoints() in Calib3d.

asked 2017-09-04 05:13:37 -0500

stephen-mhc gravatar image

I'm using OpenCV 3.0.0 for Android. Basically, I'm trying to use the chessboard pattern to find the pose of the smartphone camera, and to simply display x, y, z axes on the chessboard pattern.

To sum up,

  1. I approximate all internal camera parameters by assuming (1) focal length = image width in pixels, (2) principal point = center of image, (3) no radial distortion.
  2. I have my arbitrary world coordinate system, which is centered at the lower left corner of the chessboard pattern and have the chessboard pattern lying on x-y-plane, and have all the chessboard corner coordinates created.
  3. In every video frame, I use Calib3d.findChessboardCorners() to locate all chessboard corners in the 2D frame.
  4. Then, I use Calib3d.solvePnP() to solve for the pose, and Calib3d.projectPoints() to project several points into the current 2D frame.

However, the call to Calib3d.projectPoints() causes an error

CvException [org.opencv.core.CvException: cv::Exception: /home/maksim/workspace/android-pack/opencv/modules/calib3d/src/fisheye.cpp:77: error: (-215) _tvec.total() * _tvec.channels() == 3 && (_tvec.depth() == CV_32F || _tvec.depth() == CV_64F) in function void cv::fisheye::projectPoints(cv::InputArray, cv::OutputArray, cv::InputArray, cv::InputArray, cv::InputArray, cv::InputArray, double, cv::OutputArray)

I know some other people have also encountered such error, either in their own project or in the sample project Camera Calibration provided by OpenCV. But there's not much update from the community, so I try to ask again while giving slightly more info.

Below is the code snippet for the two overridden methods onCameraViewStarted() and onCameraFrame().

@Override
public void onCameraViewStarted(int width, int height) {
    // Initialization: Mat
    mImg = new Mat(width, height, CvType.CV_8UC4); // Current input frame
    mImgOut = new Mat(width, height, CvType.CV_8UC4); // Output frame
    mCameraMat = new Mat(3, 3, CvType.CV_32FC1); // The camera matrix

    // Approximate camera intrinsics
    mFocalLength = width; // Approximate focal length, i.e. image width in no. of pixels
    mCenter = new Point(width / 2, height / 2); // Approximate principal point, i.e. center of image

    // Construct camera matrix
    mCameraMat.put(0, 0, mFocalLength); mCameraMat.put(0, 1, 0); mCameraMat.put(0, 2, mCenter.x);
    mCameraMat.put(1, 0, 0); mCameraMat.put(1, 1, mFocalLength); mCameraMat.put(1, 2, mCenter.y);
    mCameraMat.put(2, 0, 0); mCameraMat.put(2, 1, 0); mCameraMat.put(2, 2, 1);

    // Assume no lens distortion
    mDistCoeffs = new MatOfDouble(0, 0, 0, 0); // Distortion coefficients

    // Create coordinates for all chessboard corners in the world space
    createChessboardPoints();
}

@Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    mImg = inputFrame.rgba(); // Current input frame
    mImgOut = mImg.clone(); // Output frame

    MatOfPoint2f boardCorners = new MatOfPoint2f(); // Chessboard corners in the current 2D frame

    // Find the image coordinates of all chessboard corners in the current frame
    boolean cornersFound;
    cornersFound = Calib3d.findChessboardCorners(mImg, mBoardSize, boardCorners, Calib3d.CALIB_CB_ADAPTIVE_THRESH | Calib3d.CALIB_CB_NORMALIZE_IMAGE);

    if (cornersFound) {
        // The rotation and translation vectors (the pose)
        Mat rVec = new Mat(), tVec = new Mat();

        // Solve for the pose
        Calib3d.solvePnP(mChessboardPoints, boardCorners, mCameraMat, mDistCoeffs, rVec, tVec, false, Calib3d.SOLVEPNP_ITERATIVE);

        // Several 3D points to be projected to the 2D image
        MatOfPoint3f axisTips ...
(more)
edit retag flag offensive close merge delete

Comments

aw, feck, it's still calling fisheye functions ? if so, apart from your current syntax problems, the calibration will be all wrong.

berak gravatar imageberak ( 2017-09-04 10:19:12 -0500 )edit