How to use camera parameters to calculate world distance in cm between pixels?

asked 2018-09-09 12:49:42 -0500

earcz gravatar image

I calibrated my camera using, separately, ROS, OpenCV and Matlab. People say that I need extrinsic parameters to calculate real distance between pixels in cm from the image. ROS does not provide extrinsic parameters explicitly, it provides a (4,3) projection matrix which is the output of multiplied intrinsic and extrinsic parameters.

That is why I again calibrated my camera using OpenCV and Matlab to get extrinsic parameters. Although I searched how can I calculate real distance in cm between pixels (i.e from (x1,y1) to (x2,y2)), I could not figure out how to calculate the real distance. Moreover, I did not understand that which parameters to use for distance calculation. I want to use OpenCV to calculate the distance between pixels and write the output as a txt file so that I can use this txt file to move my robot. For example, here is arrays of pixel output sample for the path,

array([[  4.484375  , 799.515625  ],
       [ 44.484375  , 487.        ],
       [255.296875  , 476.68261719],
       [267.99707031, 453.578125  ],
       [272.484375  , 306.        ],
       [403.484375  , 300.515625  ],
       [539.484375  , 296.515625  ],
       [589.421875  , 270.00292969],
       [801.109375  , 275.18554688],
       [819.        , 467.515625  ]])

I want to find the real distance in cm between these pixels in the same order. OpenCv Code that calculating parameters (taken from the official documentation of opencv)

import numpy as np
import cv2
import glob

# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

cbrow = 6
cbcol = 9

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((cbrow*cbcol,3), np.float32)
objp[:,:2] = np.mgrid[0:cbcol,0:cbrow].T.reshape(-1,2)

# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.

images = glob.glob('C:\Users\Ender\Desktop\CalibrationPhotos\*.jpg')

for fname in images:
    img = cv2.imread(fname)

    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

    # Find the chess board corners
    ret, corners = cv2.findChessboardCorners(gray, (cbcol,cbrow),None)

    # If found, add object points, image points (after refining them)
    if ret == True:
        print "%s: success" % fname
        objpoints.append(objp)

        cv2.cornerSubPix(gray,corners,(11,11),(-1,-1),criteria)
        imgpoints.append(corners)

        # Draw and display the corners
        cv2.drawChessboardCorners(img, (cbcol,cbrow), corners,ret)
        cv2.imshow('img',img)
        cv2.waitKey(150)

    else:
        print "%s: failed" % fname
        cv2.imshow('img',img)
        cv2.waitKey(1)

cv2.destroyAllWindows()

ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)

print "mtx"
print mtx
print "dist"
print dist
#print "rvecs"
#print rvecs
#print "tvecs"
#print tvecs
np.savez("CalibData.npz" ,mtx=mtx, dist=dist, rvecs=rvecs, tvecs=tvecs)

#UNDISTROTION
img = cv2.imread('C:\Users\Ender\Desktop\Maze\Maze Images\Partial Maze Images-Raw\Raw7.jpg')
h,  w = img.shape[:2]
newcameramtx, roi=cv2.getOptimalNewCameraMatrix(mtx,dist,(w,h),1,(w,h))

dst = cv2.undistort(img, mtx, dist, None, newcameramtx)

# crop the image
x,y,w,h = roi
dst = dst[y:y ...
(more)
edit retag flag offensive close merge delete

Comments

https://en.wikipedia.org/wiki/Pinhole...

with a single camera, you need fov and distance, to calculate the size.

(or fov and size, to calculate the distance)

berak gravatar imageberak ( 2018-09-09 13:01:39 -0500 )edit

if you are experienced about this issue, can you please give more detail for me to grasp the idea more ? By the way, it is not possible for me to know the size since I am calculating the shortest path in a maze.

earcz gravatar imageearcz ( 2018-09-09 13:04:38 -0500 )edit

maybe, it's impossible, then ?

berak gravatar imageberak ( 2018-09-09 13:12:10 -0500 )edit

what do you mean? do I have to know the size of the maze or any other object ? Isn't it enough that I have extrinsic and intrinsic parameters ?

earcz gravatar imageearcz ( 2018-09-09 13:13:44 -0500 )edit
1

So this maze, I assume it's 2D? You have to know something about the size of it for this to work. Either the locations of four points, or that it's square, or something that lets you pretend you know four points.

If you know it's square, you find all four corners, and just call them (0,0)(1,0)(0,1)(1,1). In that case the center would be (0.5,0.5) Check out the ArUco module for an example of how that is done.

Tetragramm gravatar imageTetragramm ( 2018-09-09 13:30:21 -0500 )edit

the extrinsic params are worthhless (describing the relation of the calibration images to the camera)

again please read https://en.wikipedia.org/wiki/Pinhole... carefully.

berak gravatar imageberak ( 2018-09-09 13:33:01 -0500 )edit

@Tetragramm maze is 3D in size of 10 m x 10 m x 40 cm. But drone does not see the maze completely, just partial view. Even if drone sees the whole maze, is it possible to find 4 outermost corner ? In addition, you gave me an idea. If I use a small object with its known size (i.e. a penny) and calculate the distance between pixels for this image, when I use the distance info for maze, is that right solution ? @berak Thank you, I will read it.

earcz gravatar imageearcz ( 2018-09-09 13:53:37 -0500 )edit

You really do need to read that link berak is posting. In short, for a 3D maze, you need a minimum of 4 known points in view at all times. The better you know their 3d position, the better your accuracy. If you can't get 4 known points somehow, then you can't do it.

Tetragramm gravatar imageTetragramm ( 2018-09-09 15:09:38 -0500 )edit

thank you, I will try.

earcz gravatar imageearcz ( 2018-09-09 15:12:03 -0500 )edit

@earcz

This is also a problem that I am also looking for a solution. If you have a solution, please show me.

monxarat gravatar imagemonxarat ( 2018-09-11 11:08:03 -0500 )edit