Ask Your Question
1

Assertion failure calling cv fisheye::calibrate

asked 2015-10-11 00:39:39 -0500

rni gravatar image

New to OpenCV so please bear with me. I have a 180' fisheye lens which I'm trying to calibrate using the methods in cv::fisheye. I tried to come up with an example program similar to the one that uses cv::calibrateCamera (but way less featured).

The problem I'm facing is that I get this assert failure when calling cv::fisheye::calibrate

OpenCV Error: Assertion failed (svd.w.at<double>(0) / svd.w.at<double>((int)svd.w.total() - 1) < thresh_cond) in CalibrateExtrinsics, file /opt/local/var/macports/build/_opt_mports_dports_graphics_opencv/opencv/work/opencv-3.0.0/modules/calib3d/src/fisheye.cpp, line 1379

The call itself looks like this:

            int flags = 0;                                                                  
            flags |= cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC;                                
            flags |= cv::fisheye::CALIB_CHECK_COND;                                         
            flags |= cv::fisheye::CALIB_FIX_SKEW;                                           

            fisheye::calibrate(obj_points, img_points, frame.size(), K, D,                  
                    noArray(), noArray(), flags);

I noticed that removing CALIB_CHECK_COND gets rid of the assertion but my undistorted image looks wrong. Any help is appreciated, the full code is below.

If I can get this working I'd love to have such an example somewhere in opensource, I found it hard to find much documentation for this sort of thing.

#include <string>                                                                                   

#include <opencv2/opencv.hpp>                                                                       

using namespace cv;                                                                                 
using namespace std;                                                                                

int main(int argc, char** argv) {                                                                   
    VideoCapture camera(0);                                                                         
    if (!camera.isOpened()) {                                                                       
        cout << "Failed to open camera." << std::endl;                                              
        return -1;                                                                                  
    }                                                                                               

    const char* window_name = "output";                                                             
    namedWindow(window_name, WINDOW_NORMAL);                                                        

    Mat frame;                                                                                      
    Size boardSize;                                                                                 
    boardSize.width = 9;                                                                            
    boardSize.height = 6;                                                                           

    int remaining_frames = 10;                                                                      
    Mat K;                                                                                          
    Mat D;                                                                                          
    Mat identity = Mat::eye(3, 3, CV_64F);                                                          

    vector<vector<Point2f> > img_points;                                                            
    vector<vector<Point3f> > obj_points(1);                                                         

    int width = 9;                                                                                  
    int height = 6;                                                                                 
    int sq_sz = 50;                                                                                 

    for (int i = 0; i < width; i++) {                                                               
        for (int j = 0; j < height; j++) {                                                          
            obj_points[0].push_back(Point3f(float(j * sq_sz), float(i * sq_sz), 0));                
        }                                                                                           
    }                                                                                               
    obj_points.resize(remaining_frames, obj_points[0]);

    while (1) {                                                                                     
        if (!camera.read(frame)) {                                                                  
            cout << "Failed to read frame" << std::endl;                                            
            break;                                                                                  
        }                                                                                           

        if (remaining_frames > 0) {                                                                 
            vector<Point2f> corners;                                                                
            bool found = findChessboardCorners(frame, boardSize, corners,                           
                    CALIB_CB_ADAPTIVE_THRESH|CALIB_CB_NORMALIZE_IMAGE|CALIB_CB_FAST_CHECK);         
            drawChessboardCorners(frame, boardSize, corners, found);                                
            if (found) {                                                                            
                img_points.push_back(corners);                                                      
                remaining_frames--;                                                                 
                cout << remaining_frames << " frames to calibration." << endl;                      

                if (remaining_frames == 0) {                                                        
                    cout << "Computing distortion" << endl;                                         

                    int flags = 0;                                                                  
                    flags |= cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC;                                
                    flags |= cv::fisheye::CALIB_CHECK_COND;                                         
                    flags |= cv::fisheye::CALIB_FIX_SKEW;                                           

                    fisheye::calibrate(obj_points, img_points, frame.size(), K, D,                  
                            noArray(), noArray(), flags);                                           
                    cout << "Finished computing distortion" << endl;                                
                }                                                                                   
            }                                                                                       

            imshow(window_name, frame);                                                             
        } else {                                                                                    
            Mat output;                                                                             
            fisheye::undistortImage(frame, output, K, D, identity);                                 
            imshow(window_name, output);                                                            
        }                                                                                           

        if (waitKey(30) > 0) {                                                                      
            break;                                                                                  
        }                                                                                           
    }                                                                                               

    return 0;                                                                                       
}
edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
1

answered 2015-12-03 12:01:09 -0500

abgg gravatar image

No need to shift the origin to the center of the chessboard for object points.

The problem was just with the for loops. Milosz has changed the order for the for loops which made it correct as per opencv order. First go across the height and then across the width.

I tried subtracting the half of width and height (which actually was not done by Milosz in his code as can be seen clearly) but it did not work. Then I noticed the for loop and tried the for loop by Milosz without subtracting half of the width and heights and it worked without the assertion.

edit flag offensive delete link more
0

answered 2017-07-10 02:54:36 -0500

willhua gravatar image

There is another solution: Reduce the number of imagesused to calibration. http://answers.opencv.org/question/10...

The max count of images during my test is 6.

edit flag offensive delete link more
0

answered 2015-11-03 06:57:55 -0500

Milosz gravatar image

updated 2015-11-04 06:49:25 -0500

I have the same problem, the code runs but except for very trivial cases (one set of points, checkerboard middle of the screen) it always fails the assertion. The test case included in OCV doesn't include the input data (left.xml and object.xml files) so it is hard to figure out what is wrong in my input data.

Edit: I think i have nailed it. The problem is the 3D object point set, instead of having the origin (0,0) in the corner of the checkerboard like in the OpenCV examples i have put an offset to put the origin in the middle of the checkerboard by subtracting the halved checkerboard width and height. Below my code to construct the 3d object points:

std::vector<cv::Point3d> Create3DChessboardCorners(cv::Size boardSize, float squareSize)
{
  // This function creates the 3D points of your chessboard in its own coordinate system
float width = (boardSize.width-1)*squareSize;
float height = (boardSize.height-1)*squareSize;


  std::vector<cv::Point3d> corners;

  for( int i = 0; i < boardSize.height; i++ )
  {
    for( int j = 0; j < boardSize.width; j++ )
    {
      corners.push_back(cv::Point3d(float(j*squareSize)-width, float(i*squareSize)-height, 0));
    }
  }

  return corners;
}

Now it seeim to be working, I have tested it by calculating the pose (rotation and translation) of the pattern by cv::solvePnP and projecting back to 2d by cv::fisheye::projectPoints

edit flag offensive delete link more

Comments

For those who want to look over the test data, you can download all test data and files from github. See http://answers.opencv.org/question/45...

Chang gravatar imageChang ( 2017-05-02 04:56:49 -0500 )edit
Login/Signup to Answer

Question Tools

3 followers

Stats

Asked: 2015-10-11 00:39:39 -0500

Seen: 4,722 times

Last updated: Nov 04 '15