Stereo Vision (Python) - How can I improve my Stereo Rectification results?

asked 2017-03-26 10:17:01 -0500

Lexus09 gravatar image

updated 2017-03-26 10:22:30 -0500

Hi all,

I am carrying out an undergraduate where I need to find out the depth of a stereo image, thus it is crucial to get a good disparity map for the calculations to be accurate. My rectification results are pretty mediocre at best and I have carried out the calibration countless times with no success, only minimal variations between results. I would highly appreciate experts' guidance on this issue. Like, can I try changing some parameters of my stereoRectify method? I already tried to tinker with the stereocalibrate parameters with minimal success.

I cannot post my results on this question as I have insufficient karma, but the rectified images look as if they were warped in a spiral manner and overriding the original video capture. Copy this in the URL if you wish to see a sample of my results: scontent.fmla1-2.fna.fbcdn.net/v/t35.0-12/17571001_10212528106338046_845875913_o.png?oh=3ef04efa50a3d9f4e4d253f2f57ae298&oe=58DAF735

This is my code for the calibration:

import cv,time,sys
import cv2
from itertools import izip
# python calibrate.py board_w,board_h and boards_n should be given by the user
n_boards=0  #no of boards
board_w=int(sys.argv[1])    # number of horizontal corners
board_h=int(sys.argv[2])    # number of vertical corners
n_boards=int(sys.argv[3])   # no. of views passed as pictures
board_n=board_w*board_h     # no of total corners
board_sz=(board_w,board_h)  # size of board

#   creation of memory storages for left camera
image_points0=cv.CreateMat(n_boards*board_n,2,cv.CV_32FC1)
object_points=cv.CreateMat(n_boards*board_n,3,cv.CV_32FC1)
point_counts=cv.CreateMat(n_boards,1,cv.CV_32SC1)
intrinsic_matrix0=cv.CreateMat(3,3,cv.CV_32FC1)
distortion_coefficient0=cv.CreateMat(5,1,cv.CV_32FC1)

#   creation of memory storages for right camera
image_points1=cv.CreateMat(n_boards*board_n,2,cv.CV_32FC1)
intrinsic_matrix1=cv.CreateMat(3,3,cv.CV_32FC1)
distortion_coefficient1=cv.CreateMat(5,1,cv.CV_32FC1)

R=cv.CreateMat(3,3,cv.CV_64F)
T=cv.CreateMat(3,1,cv.CV_64F)
E=cv.CreateMat(3,3,cv.CV_64F)
F=cv.CreateMat(3,3,cv.CV_64F)
#term_crit=(cv.CV_TERMCRIT_EPS+cv.CV_TERMCRIT_ITER,30,0.1)
#capture frames of specified properties and modification of matrix values
i=0
z=0     # to print number of frames
successes=0


#   capturing required number of views

found=0
with open('Left.txt','rb') as f:
    imgl= [line.strip() for line in f]
    with open('Right.txt','rb') as f1:
         imgr=[line.strip() for line in f1]
         for (image0,image1) in izip(imgl,imgr):
        if image0 =='' or image1== '' :
            break;
        else:

                Image0 = cv.LoadImage(image0)
                Image1 = cv.LoadImage(image1)

            gray_image0=cv.CreateImage(cv.GetSize(Image0),8,1)
            cv.CvtColor(Image0,gray_image0,cv.CV_BGR2GRAY)
            gray_image1=cv.CreateImage(cv.GetSize(Image1),8,1)
            cv.CvtColor(Image1,gray_image1,cv.CV_BGR2GRAY)
            (found0,corners0)=cv.FindChessboardCorners(gray_image0,board_sz,cv.CV_CALIB_CB_ADAPTIVE_THRESH| cv.CV_CALIB_CB_FILTER_QUADS)
            (found1,corners1)=cv.FindChessboardCorners(gray_image1,board_sz,cv.CV_CALIB_CB_ADAPTIVE_THRESH| cv.CV_CALIB_CB_FILTER_QUADS)
            cv.FindCornerSubPix(gray_image0,corners0,(11,11),(-1,-1),(cv.CV_TERMCRIT_EPS+cv.CV_TERMCRIT_ITER,30,0.001))
            cv.FindCornerSubPix(gray_image1,corners1,(11,11),(-1,-1),(cv.CV_TERMCRIT_EPS+cv.CV_TERMCRIT_ITER,30,0.001))     
            # if got a good ...
(more)
edit retag flag offensive close merge delete

Comments

Intrinsic parameters seems wrong. Don't you use calibrateCamera to have a good guess of intrinsic parameters?

LBerger gravatar imageLBerger ( 2017-03-26 10:48:42 -0500 )edit

Thank you for your comment, the thing is I am using the stereoCalibrate method, isn't that the same thing just for stereo cameras?

Lexus09 gravatar imageLexus09 ( 2017-03-26 10:57:17 -0500 )edit

No StereoCalibrate is a method to estimate E, F R and T and find a better estimation of intrinsic parameter. When you use StereoCalibrate as you do 11 parameters (R,T) and 4 parameters (intrinsic) and 5 parameters for distortion must be estimated. It is difficult.

It is better to first estimate intrinsic and distortion parameters for each camera and then call StereoCalibrate using INTRINSIC_GUESS parameters.

Do you really need K4, K5, K6 parameters? If distortion is not visible you can set these parameters to 0

LBerger gravatar imageLBerger ( 2017-03-26 12:14:51 -0500 )edit

Can you guide me on how to do this? As I am pretty new to OpenCV/stereo vision in general. If not, thanks a million for your time and help, I really appreciate anyways :)

Lexus09 gravatar imageLexus09 ( 2017-03-26 12:20:56 -0500 )edit

Sorry I don't know python but you can use this tutorial to calibrate left and right camera. Try it and give results to StereoCalibrate.

LBerger gravatar imageLBerger ( 2017-03-26 13:35:38 -0500 )edit
1

Ok thank you! :)

Lexus09 gravatar imageLexus09 ( 2017-03-26 14:05:57 -0500 )edit

please use cv2, the arcane cv api got removed with opencv3 !

(this is all horribly outdated, and should NEVER be used)

berak gravatar imageberak ( 2017-03-27 00:35:18 -0500 )edit

Do you think it might affect the results obtained??

Lexus09 gravatar imageLexus09 ( 2017-03-27 13:06:10 -0500 )edit

^^ not really, but you won't be able to run that code on current opencv (they removed that api entirely)

please do not write more DEAD code, please do not fool noobs seing this into thinking, it would be a valid idea. (stackoverflow has done a lot of harm there already ..)

berak gravatar imageberak ( 2017-03-27 13:21:07 -0500 )edit