How to find the location of a point in an image in millimeters using camera matrix?

asked 2020-10-23 02:06:21 -0500

Tehseen gravatar image

updated 2020-10-23 02:07:56 -0500

I am using a standard 640x480 webcam. I have done Camera calibration in OpenCV in Python 3. This the Code I am using. The code is working and giving me the Camera Matrix and Distortion Coefficients successfully. Now, How can I find how many millimeters are there in 640 pixels in my scene image. I have attached the webcam above a table horizontally and on the table, a Robotic arm is placed. Using the camera I am finding the centroid of an object. Using Camera Matrix my goal is to convert the location of that object (e.g. 300x200 pixels) to the millimeter units so that I can give the millimeters to the robotic arm to pick that object. I have searched but not find any relevant information. Please tell me that is there any equation or method for that. Thanks a lot!

This is my Code:

import numpy as np
import cv2
import yaml
import os

# Parameters
#TODO : Read from file
n_row=4  #Checkerboard Rows
n_col=6  #Checkerboard Columns
n_min_img = 10 # number of images needed for calibration
square_size = 40  # size of each individual box on Checkerboard in mm  
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001) # termination criteria
corner_accuracy = (11,11)
result_file = "./calibration.yaml"  # Output file having camera matrix

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(n_row-1,n_col-1,0)
objp = np.zeros((n_row*n_col,3), np.float32)
objp[:,:2] = np.mgrid[0:n_row,0:n_col].T.reshape(-1,2) * square_size

# Intialize camera and window
camera = cv2.VideoCapture(0) #Supposed to be the only camera
if not camera.isOpened():
    print("Camera not found!")
    quit()
width = int(camera.get(cv2.CAP_PROP_FRAME_WIDTH))  
height = int(camera.get(cv2.CAP_PROP_FRAME_HEIGHT))
cv2.namedWindow("Calibration")


# Usage
def usage():
    print("Press on displayed window : \n")
    print("[space]     : take picture")
    print("[c]         : compute calibration")
    print("[r]         : reset program")
    print("[ESC]    : quit")

usage()
Initialization = True

while True:    
    if Initialization:
        print("Initialize data structures ..")
        objpoints = [] # 3d point in real world space
        imgpoints = [] # 2d points in image plane.
        n_img = 0
        Initialization = False
        tot_error=0

    # Read from camera and display on windows
    ret, img = camera.read()
    cv2.imshow("Calibration", img)
    if not ret:
        print("Cannot read camera frame, exit from program!")
        camera.release()        
        cv2.destroyAllWindows()
        break

    # Wait for instruction 
    k = cv2.waitKey(50) 

    # SPACE pressed to take picture
    if k%256 == 32:   
        print("Adding image for calibration...")
        imgGray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

        # Find the chess board corners
        ret, corners = cv2.findChessboardCorners(imgGray, (n_row,n_col),None)

        # If found, add object points, image points (after refining them)
        if not ret:
            print("Cannot found Chessboard corners!")

        else:
            print("Chessboard corners successfully found.")
            objpoints.append(objp)
            n_img +=1
            corners2 = cv2.cornerSubPix(imgGray,corners,corner_accuracy,(-1,-1),criteria)
            imgpoints.append(corners2)

            # Draw and display the corners
            imgAugmnt = cv2.drawChessboardCorners(img, (n_row,n_col), corners2,ret)
            cv2.imshow('Calibration',imgAugmnt) 
            cv2.waitKey(500)        

    # "c" pressed to compute calibration        
    elif k%256 == 99:        
        if n_img <= n_min_img:
            print("Only ", n_img , " captured, ",  " at least ", n_min_img , " images are needed")

        else:
            print("Computing calibration ...")
            ret, mtx ...
(more)
edit retag flag offensive close merge delete