# undistort() and undistortPoints() are inconsistent

For testing I generate a grid image as matrix and again the grid points as point array:

This represents a "distorted" camera image along with some feature points. When I now undistort both the image and the grid points, I get the following result:

(Note that the fact that the "distorted" image is straight and the "undistorted" image is morphed is not the point, I'm just testing the undistortion functions with a straight test image.)

The grid image and the red grid points are totally misaligned now. I googled and found that some people forget to specify the "new camera matrix" parameter in undistortPoints but I didn't. The documentation also mentions a normalization but I still have the problem when I use the identity matrix as camera matrix. Also, in the central region it fits perfectly.

Why is this not identical, do I use something in a wrong way?

I use cv2 (4.1.0) in Python. Here is the code for testing:

import numpy as np
import matplotlib.pyplot as plt
import cv2

w = 401
h = 301

# helpers
#--------

def plotImageAndPoints(im, pu, pv):
plt.imshow(im, cmap="gray")
plt.scatter(pu, pv, c="red", s=16)
plt.xlim(0, w)
plt.ylim(0, h)
plt.show()

def cv2_undistortPoints(uSrc, vSrc, cameraMatrix, distCoeffs):
uvSrc = np.array([np.matrix([uSrc, vSrc]).transpose()], dtype="float32")
uvDst = cv2.undistortPoints(uvSrc, cameraMatrix, distCoeffs, None, cameraMatrix)
uDst = [uv[0] for uv in uvDst[0]]
vDst = [uv[1] for uv in uvDst[0]]
return uDst, vDst

# test data
#----------

# generate grid image
img = np.ones((h, w), dtype = "float32")
img[0::20, :] = 0
img[:, 0::20] = 0

# generate grid points
uPoints, vPoints = np.meshgrid(range(0, w, 20), range(0, h, 20), indexing='xy')
uPoints = uPoints.flatten()
vPoints = vPoints.flatten()

# see if points align with the image
plotImageAndPoints(img, uPoints, vPoints) # perfect!

# undistort both image and points individually
#---------------------------------------------

# camera matrix parameters
fx = 1
fy = 1
cx = w/2
cy = h/2

# distortion parameters
k1 = 0.00003
k2 = 0
p1 = 0
p2 = 0

# convert for opencv
mtx = np.matrix([
[fx,  0, cx],
[ 0, fy, cy],
[ 0,  0,  1]
], dtype = "float32")

dist = np.array([k1, k2, p1, p2], dtype = "float32")

# undistort image
imgUndist = cv2.undistort(img, mtx, dist)
# undistort points
uPointsUndist, vPointsUndist = cv2_undistortPoints(uPoints, vPoints, mtx, dist)

# test if they still match
plotImageAndPoints(imgUndist, uPointsUndist, vPointsUndist) # awful!


Any help appreciated!

edit retag close merge delete

Sort by » oldest newest most voted

To undistort your image, what is probably done:

• iterate over pixels in the destination image
• for (x,y) in the destination image, compute the distorted pixel coordinate (x',y')
• interpolate the pixel intensity at (x',y')
• assign at (x,y) the interpolated pixel intensity at (x',y')

To undistort the pixel coordinate, this is another formula. Something like iteratively undistort the pixel coordinate. Remember that the distortion function is non-linear.

more

Thank you! I am aware that these are two different procedures. But if one of the two (my bet is on undistortPoints) can be that inaccurate, this should be documented somewhere... It took me forever to discover that the inconsistency in my data actually originates from OpenCV. If people use this to do 3D reconstruction based on detected features, the results will also be totally off. When I find the time, I will do some more investigation.

( 2020-06-04 09:52:35 -0500 )edit

undistortPoints() implementation and interface can probably be improved.

But the important part is that it is a non-linear problem. Stating that, depending on how well the function is implemented, you can diverge, or not accurately converge.

( 2020-06-06 11:43:34 -0500 )edit

Official site

GitHub

Wiki

Documentation