Ask Your Question
3

stereo calibration issues

asked 2016-07-30 18:54:35 -0500

Yolteotl gravatar image

updated 2016-07-31 08:42:06 -0500

Hi!

I'm currently working on a little robot with a stereo setup (made with 2 logitech C270, not the best, but still decent with 640x480 on linux) to detect obstacles indoor and create a 2D maps of the room.

I tried to calibrate both cameras then calibrate the whole setup and I achieved finally to get decent results :

Left camera with 100 images :

errReproj = 0.16552850392837773

cameraMatrix = 
[813,6236504501936, 0, 322,084934872842;
0, 813,7875223412764, 246,1504964804209;
 0, 0, 1]

distCoeffs =[0,01298432313476952; 0,06663299254719597;
 0;
 0;
 -1,474057784906827]

Right camera with 100 images :

errReproj = 0.1717557062634702

cameraMatrix = 
[812,5880346983992, 0, 311,5146085418734;
 0, 813,0827212604557, 232,3997472460254;
 0, 0, 1]

distCoeffs = 
[0,01009600201848234;
-0,2462621231670898;
0;
0;
 1,891318564864029]

Stereo calibration :

RMS=0.3906354057793356

My number seems decent, even if the stereo calibration could be a bit better. I'm using those options for the camera calibration :

    private static final int flagsCorner = Calib3d.CALIB_CB_FAST_CHECK 
            | Calib3d.CALIB_CB_ADAPTIVE_THRESH
            | Calib3d.CALIB_CB_NORMALIZE_IMAGE;
    private static final int flagsCalib = Calib3d.CALIB_CB_FAST_CHECK;
    private static final TermCriteria criteria = new TermCriteria(TermCriteria.EPS
            + TermCriteria.COUNT, 5000000, 0.00000001);

And those ones for the stereo :

private static final int CORNERS_FLAGS = Calib3d.CALIB_CB_FAST_CHECK 
    | Calib3d.CALIB_CB_ADAPTIVE_THRESH
    | Calib3d.CALIB_CB_NORMALIZE_IMAGE;
private static final int CALIB_FLAGS = Calib3d.CALIB_CB_FAST_CHECK;
private static final TermCriteria CORNERS_CRITERIA = new TermCriteria(TermCriteria.EPS
    + TermCriteria.COUNT, 5000000, 0.00000001);
private static final TermCriteria CALIB_CRITERIA = new TermCriteria(TermCriteria.EPS
    + TermCriteria.COUNT, 5000000, 0.0001);

So, my issue is that I can't get a decent disparity map. Even if the image seems to be undistored well :

Original pic :

left original right original

After undistortion + adding line to show Y axis :

left undistorded right undistorded

But when it goes through the StereoBM.compute(), I can't get a good disparity map, even by tweaking the different values available :

disparity map

This is an example with those parameters :

Blocksize : 39
Smaller Blocksize : 0
Disp12MaxDiff : 0
Min disparity : 1
Num of disparity : 32
Prefilter cap : 61
Prefilter size : 5
Speckle Range : 0
Speckle window size : 0
Texture threshold : 74
Uniqueness Ratio : 6

So, I would like to know if something looks wrong in my results, matrix, and datas, any help is welcome!

Thank you

EDIT : I made more tests this morning on a more opened area, and I'm getting better results : new results

It looks like I'm able to get mid to long range datas (~4/5m and more) but It is struggling with low range. How can I improve this? Is it a calibration issue? The distance between both cameras is currently of 14cm. Is it too much?

My goal would be to let the robot navigating among the room, so mid/long range detection is not really important, but close detection is!

EDIT2: After reading internet stuff ( https://www.cs.cf.ac.uk/Dave/Vision_l... ), I understood that my setup (14cm between both cameras) was not good to find close items. I rebuilt my robot so I only have 7cm between both cameras and it looks much better!

My first calibration gave me a RMS of 0.42, down to 0.28 ... (more)

edit retag flag offensive close merge delete

Comments

How did you do the intrinsic calibration of the cameras? I've never seen coefficients like yours.

FooBar gravatar imageFooBar ( 2016-07-31 05:28:21 -0500 )edit

Hi, I converted a C++ sample in Java : http://warcraft3campagne.free.fr/Cali...

Then I took 100 pics with a python script detecting chessboards.

Yolteotl gravatar imageYolteotl ( 2016-07-31 06:05:15 -0500 )edit

Hey You. Can you share about your code? (If you can)

hoang anh tuan gravatar imagehoang anh tuan ( 2017-01-15 21:18:25 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
4

answered 2016-07-31 07:43:40 -0500

updated 2016-07-31 08:07:48 -0500

There is at least a problem with your intrinsic calibration. Two of your entries are exactly zero which looks suspicious. The problem here is the flag you pass to calibrateCamera. You pass 'CALIB_CB_FAST_CHECK' which is a flag for the checkerboard detection. It's value is 8 (http://docs.opencv.org/trunk/d9/d0c/g...). calibrateCamera however only expects values that are listed here: http://docs.opencv.org/2.4/modules/ca...

In the list we find

cv::CALIB_ZERO_TANGENT_DIST = 0x00008,

which forces your tangential distortion (the second and third entry (your zeros)) to zero.

Could you rerun the intrinsic calibration without this flag? This should decrease the calibration's RMSE and maybe also improve your final result.

BTW: It's great that you put that much information into the question! This helps a lot to find such a subtle bug.

You should also have a look at your stereoCalibrate-Flags. You again pass only 'CV_CALIB_FIX_INTRINSIC' so that you re-estimate your intrinsic calibration (I assume that you use cv::stereoCalibrate). As you already know your intrinsic calibration, just pass 'CV_CALIB_FIX_INTRINSIC' to only estimate relative pose of the cameras. I guess you use a lot less images for that so that your estimate for the intrinsics are really bad.

edit flag offensive delete link more

Comments

1

This is a great answer too :) I was suspecting that my setup was not good enough to detect close items, I rebuilt my robot to have only 7cm between both cameras (instead of 14) and it seems MUCH better. (I will edit my post with results) After reading some stuff on internet ( here : https://www.cs.cf.ac.uk/Dave/Vision_l... ), it looks that it was a good thing to do.

I tried the CV_CALIB_FIX_INTRINSIC but the RMS jumped to >2000. Right now, I'm keeping CV_CALIB_SAME_FOCAL_LENGTH and CV_CALIB_ZERO_TANGENT_DIST, the RMS is down to 0.28!

Yolteotl gravatar imageYolteotl ( 2016-07-31 08:35:16 -0500 )edit

Can you show how you call the stereoCalibrate? How do you pass the calibration of the cameras?

FooBar gravatar imageFooBar ( 2016-07-31 09:48:04 -0500 )edit

Yes of course : http://warcraft3campagne.free.fr/Ster... I compute calibration of both cameras, then I undistort the images I use for the stereo calibration and pass them through stereoCalibrate

Yolteotl gravatar imageYolteotl ( 2016-07-31 10:47:51 -0500 )edit

Is this your current code? And please edit the question so that this answer can still be understood after your links expired. In the stereoCalibrate, you still pass the CALIB_CB_FAST_CHECK. And I don't understand your logic. You first load leftCalibration and rightCalibration and use it to undistort the images. But then you don't pass them to stereoCalibrate but re-estimates them. Why do you do that?

FooBar gravatar imageFooBar ( 2016-07-31 11:10:08 -0500 )edit

Oh, nice catch, I did not even notice there was a stereoCalibrate call with both cameras calibration matrix... It gives a little improvement! I'm new to OpenCV and "just" need it to complete an obstacle map of the room. (I will also add an handmade sonar using a compass + IR range sensor)

Yolteotl gravatar imageYolteotl ( 2016-07-31 14:08:56 -0500 )edit
Login/Signup to Answer

Question Tools

4 followers

Stats

Asked: 2016-07-30 18:54:35 -0500

Seen: 1,227 times

Last updated: Jul 31 '16