SolvePnP detection errors [ios]

asked 2013-11-07 18:50:27 -0500

axadiw gravatar image

updated 2013-11-19 19:21:21 -0500


I've got problem with precise detection of markers using OpenCV.

I've recorded video presenting that issue:

As you see I'm markers that I'm detecting are slightly moved at some camera angles. I've read on the web that this may be camera calibration problems, so I'll tell you guys how I'm calibrating camera, and maybe you'd be able to tell me what am I doing wrong?

At the beginnig I'm collecting data from various images, and storing calibration corners in _imagePoints vector like this

std::vector<cv::Point2f> corners;
_imageSize = cvSize(image->size().width, image->size().height);

bool found = cv::findChessboardCorners(*image, _patternSize, corners);

if (found) {
    cv::Mat *gray_image = new cv::Mat(image->size().height, image->size().width, CV_8UC1);
    cv::cvtColor(*image, *gray_image, CV_RGB2GRAY);

    cv::cornerSubPix(*gray_image, corners, cvSize(11, 11), cvSize(-1, -1), cvTermCriteria(CV_TERMCRIT_EPS+ CV_TERMCRIT_ITER, 30, 0.1));

    cv::drawChessboardCorners(*image, _patternSize, corners, found);


Than, after collecting enough data I'm calculating camera matrix and coefficients with this code:

std::vector< std::vector<cv::Point3f> > *objectPoints = new std::vector< std::vector< cv::Point3f> >();

for (unsigned long i = 0; i < _imagePoints->size(); i++) {
    std::vector<cv::Point2f> currentImagePoints = _imagePoints->at(i);
    std::vector<cv::Point3f> currentObjectPoints;

    for (int j = 0; j < currentImagePoints.size(); j++) {
        cv::Point3f newPoint = cv::Point3f(j % _patternSize.width, j / _patternSize.width, 0);



std::vector<cv::Mat> rvecs, tvecs;

static CGSize size = CGSizeMake(_imageSize.width, _imageSize.height);
cv::Mat cameraMatrix = [_userDefaultsManager cameraMatrixwithCurrentResolution:size]; // previously detected matrix
cv::Mat coeffs = _userDefaultsManager.distCoeffs; // previously detected coeffs
cv::calibrateCamera(*objectPoints, *_imagePoints, _imageSize, cameraMatrix, coeffs, rvecs, tvecs);

Results are like you've seen in the video.

What am I doing wrong? is that an issue in the code? How much images should I use to perform calibration (right now I'm trying to obtain 20-30 images before end of calibration).

Should I use images that containg wrongly detected chessboard corners, like this:

photo 1

or should I use only properly detected chessboards like these:

photo 2 photo 3

I've been experimenting with circles grid instead of of chessboards, but results were much worse that now.

In case of questions how I'm detecting marker: I'm using solvepnp function:

solvePnP(modelPoints, imagePoints, [_arEngine currentCameraMatrix], _userDefaultsManager.distCoeffs, rvec, tvec);

with modelPoints specified like this:

    markerPoints3D.push_back(cv::Point3d(-kMarkerRealSize / 2.0f, -kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(kMarkerRealSize / 2.0f, -kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(kMarkerRealSize / 2.0f, kMarkerRealSize / 2.0f, 0));
    markerPoints3D.push_back(cv::Point3d(-kMarkerRealSize / 2.0f, kMarkerRealSize / 2.0f, 0));

and imagePoints are coordinates of marker corners in processing image (I'm using custom algorithm to do that)

edit retag flag offensive close merge delete


I've posted some thoughts about this question here:

It looks like it's not callibration problem, I still don't know what's causing it

axadiw gravatar imageaxadiw ( 2013-11-19 19:23:36 -0500 )edit