Ask Your Question
3

undistort() and undistortPoints() are inconsistent

asked 2020-06-03 08:37:35 -0500

mqnc gravatar image

For testing I generate a grid image as matrix and again the grid points as point array:

"distorted" camera image with feature points

This represents a "distorted" camera image along with some feature points. When I now undistort both the image and the grid points, I get the following result:

image and points after individual undistortion zoom into undistorted result

(Note that the fact that the "distorted" image is straight and the "undistorted" image is morphed is not the point, I'm just testing the undistortion functions with a straight test image.)

The grid image and the red grid points are totally misaligned now. I googled and found that some people forget to specify the "new camera matrix" parameter in undistortPoints but I didn't. The documentation also mentions a normalization but I still have the problem when I use the identity matrix as camera matrix. Also, in the central region it fits perfectly.

Why is this not identical, do I use something in a wrong way?

I use cv2 (4.1.0) in Python. Here is the code for testing:

import numpy as np
import matplotlib.pyplot as plt
import cv2

w = 401
h = 301


# helpers
#--------

def plotImageAndPoints(im, pu, pv):
    plt.imshow(im, cmap="gray")
    plt.scatter(pu, pv, c="red", s=16)
    plt.xlim(0, w)
    plt.ylim(0, h)
    plt.show()

def cv2_undistortPoints(uSrc, vSrc, cameraMatrix, distCoeffs):
    uvSrc = np.array([np.matrix([uSrc, vSrc]).transpose()], dtype="float32")
    uvDst = cv2.undistortPoints(uvSrc, cameraMatrix, distCoeffs, None, cameraMatrix)
    uDst = [uv[0] for uv in uvDst[0]]
    vDst = [uv[1] for uv in uvDst[0]]
    return uDst, vDst


# test data
#----------

# generate grid image
img = np.ones((h, w), dtype = "float32")
img[0::20, :] = 0
img[:, 0::20] = 0

# generate grid points
uPoints, vPoints = np.meshgrid(range(0, w, 20), range(0, h, 20), indexing='xy')
uPoints = uPoints.flatten()
vPoints = vPoints.flatten()

# see if points align with the image
plotImageAndPoints(img, uPoints, vPoints) # perfect!


# undistort both image and points individually
#---------------------------------------------

# camera matrix parameters
fx = 1
fy = 1
cx = w/2
cy = h/2

# distortion parameters
k1 = 0.00003
k2 = 0
p1 = 0
p2 = 0

# convert for opencv
mtx = np.matrix([
    [fx,  0, cx],
    [ 0, fy, cy],
    [ 0,  0,  1]
], dtype = "float32")

dist = np.array([k1, k2, p1, p2], dtype = "float32")

# undistort image
imgUndist = cv2.undistort(img, mtx, dist)
# undistort points
uPointsUndist, vPointsUndist = cv2_undistortPoints(uPoints, vPoints, mtx, dist)

# test if they still match
plotImageAndPoints(imgUndist, uPointsUndist, vPointsUndist) # awful!

Any help appreciated!

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-06-04 05:06:36 -0500

Eduardo gravatar image

To undistort your image, what is probably done:

  • iterate over pixels in the destination image
  • for (x,y) in the destination image, compute the distorted pixel coordinate (x',y')
  • interpolate the pixel intensity at (x',y')
  • assign at (x,y) the interpolated pixel intensity at (x',y')

To undistort the pixel coordinate, this is another formula. Something like iteratively undistort the pixel coordinate. Remember that the distortion function is non-linear.

edit flag offensive delete link more

Comments

Thank you! I am aware that these are two different procedures. But if one of the two (my bet is on undistortPoints) can be that inaccurate, this should be documented somewhere... It took me forever to discover that the inconsistency in my data actually originates from OpenCV. If people use this to do 3D reconstruction based on detected features, the results will also be totally off. When I find the time, I will do some more investigation.

mqnc gravatar imagemqnc ( 2020-06-04 09:52:35 -0500 )edit

undistortPoints() implementation and interface can probably be improved.

But the important part is that it is a non-linear problem. Stating that, depending on how well the function is implemented, you can diverge, or not accurately converge.

Eduardo gravatar imageEduardo ( 2020-06-06 11:43:34 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2020-06-03 08:37:35 -0500

Seen: 71 times

Last updated: Jun 04