Ask Your Question
0

Find rotation vector centering undistorted image on specific fisheye image pixel

asked 2020-01-13 04:07:01 -0500

mart_fire gravatar image

updated 2020-01-15 07:59:17 -0500

I have calibrated a fisheye camera. To undistort image, I use initUndistortRectifyMap() followed by remap(). In initUndistortRectifyMap() I can give a rotation matrix in argument in order to rotate the undistorted image on a specific part of the fisheye image. I would like to know if there is a way to find automatically the rotation vector so that given a specific pixel position in the fisheye, the undistorted image in centered on this pixel. It could be used to track someone in the fisheye image and then undistort this person for algorithms which need an undistorted image. This the code I'm using to undistort the image.

```

def undistort(self, img,  zoom=1.0, rot_vec=np.eye(3)):
    """
    Undistort an image from a fisheye camera

    :param img: The distorted image (numpy array).
    :param alpha: (optional) Free scaling parameter between 0 (when all the pixels in the undistorted image are valid) and 1 (when all the source image pixels are retained in the undistorted image).
    :param zoom: (optional) Zoom factor in the undistorted images.
    :param rot_vec: (optional) Rotation vector (alpha, beta, gamma) used to rotate view of undistorted fisheye image.
    :return: The undistorted image.
    """
    t0 = timeit.default_timer()
    h, w = img.shape[:2]  # the dimension of input image to un-distort

    if self._cam_type == "fish_eye":

        scaled_K = self._K.copy()
        scaled_K[0, 0] *= zoom
        scaled_K[1, 1] *= zoom

        R = cv2.Rodrigues(rot_vec)[0]

        # self._K is the camera matrix end self._D the distortion coefficients
        map1, map2 = cv2.fisheye.initUndistortRectifyMap(self._K, self._D, R, scaled_K, (w, h), cv2.CV_16SC2)
        img_undistorted = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)

    return img_undistorted

```

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-01-15 08:05:08 -0500

mart_fire gravatar image

updated 2020-01-15 08:07:50 -0500

Finally, I used fsolve() from scipy using equations I got from initUndistortRectify() source code. I only get first 2 angles so one may need to rotate the image.

```

def _find_angles(self, u, v):
    """
    Finds 'alpha' and 'beta' Rodrigues' angles so that the output undistorted image is centered on a specific input
    pixel from the fisheye image.

    :param u: x-coordinate of the pixel to center.
    :param v: y-coordinate of the pixel to center.
    :return: 'alpha' and 'beta' Euler angles
    """
    tic = timeit.default_timer()

    def _equations(ab, *args):
        alpha, beta = ab
        u, v, = args
        w = self.dims[0]
        h = self.dims[1]

        f = [self.K[0, 0], self.K[1, 1]]
        c = [self.K[0, 2], self.K[1, 2]]

        P = self.K.copy()
        P[0, 0] *= self.zoom
        P[1, 1] *= self.zoom

        R = cv2.Rodrigues(np.array([alpha, beta, 0.]))[0]

        iR = np.linalg.inv(np.dot(P, R))

        _x = h/2 * iR[0, 0] + w/2 * iR[0, 1] + iR[0, 2]
        _y = h/2 * iR[1, 0] + w/2 * iR[1, 1] + iR[1, 2]
        _w = h/2 * iR[2, 0] + w/2 * iR[2, 1] + iR[2, 2]

        x = _x / _w
        y = _y / _w
        r = np.sqrt(x**2 + y**2)

        theta = np.arctan(r)
        theta_d = theta * (1 + self.D[0]*theta**2 + self.D[1]*theta**4 + self.D[2]*theta**6 + self.D[3]*theta**8)
        scale = (r == 0) * 1.0 + (1 - (r == 0)) * theta_d / r
        return f[0]*x*scale + c[0] - u, f[1]*y*scale + c[1] - v

    alpha, beta = fsolve(_equations, (0., 0.), args=(u, v))
    print(alpha, beta)
    print("Computation time :", timeit.default_timer() - tic)
    return alpha, beta

```

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-01-13 04:07:01 -0500

Seen: 23 times

Last updated: Jan 15