Revision history [back]

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[-1.206582 , -0.8626694],
        [ 1.2490402, -0.9112338],
        [-1.8511834,  1.3299862],
        [ 2.2616546,  1.5442562]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, undistorted_centers, numpy.eye(3), new_K):

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[265.85552573993016, 0.0, 320.7336534207575], [0.0, 266.30000369971407, 229.6288582732103], [0.0, 0.0, 1.0]]), 
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[1.2351158806052929, 0.0, 2268.4171643974255], [0.0, 1.2371808434650273, 1713.4398768258288], [0.0, 0.0, 1.0]])
)

array([[[2266.9248, 1712.3696],
        [2268.3188, 1713.4639],
        [2268.4397, 1713.3184],
        [2268.3857, 1713.4268]]], dtype=float32)

Okay, those look like pixel coordinates; but why are they all almost the same point? This also doesn't make sense.

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[-1.206582 , -0.8626694],
        [ 1.2490402, -0.9112338],
        [-1.8511834,  1.3299862],
        [ 2.2616546,  1.5442562]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, undistorted_centers, numpy.eye(3), new_K)R=numpy.eye(3), K=new_K):

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[265.85552573993016, 0.0, 320.7336534207575], [0.0, 266.30000369971407, 229.6288582732103], [0.0, 0.0, 1.0]]), 
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[1.2351158806052929, 0.0, 2268.4171643974255], [0.0, 1.2371808434650273, 1713.4398768258288], [0.0, 0.0, 1.0]])
)

array([[[2266.9248, 1712.3696],
        [2268.3188, 1713.4639],
        [2268.4397, 1713.3184],
        [2268.3857, 1713.4268]]], dtype=float32)

Okay, those look like pixel coordinates; but why are they all almost the same point? This also doesn't make sense.

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[-1.206582 , -0.8626694],
        [ 1.2490402, -0.9112338],
        [-1.8511834,  1.3299862],
        [ 2.2616546,  1.5442562]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, R=numpy.eye(3), K=new_K)P=new_K):

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[265.85552573993016, 0.0, 320.7336534207575], [0.0, 266.30000369971407, 229.6288582732103], [0.0, 0.0, 1.0]]), 
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[1.2351158806052929, 0.0, 2268.4171643974255], [0.0, 1.2371808434650273, 1713.4398768258288], [0.0, 0.0, 1.0]])
)

array([[[2266.9248, 1712.3696],
        [2268.3188, 1713.4639],
        [2268.4397, 1713.3184],
        [2268.3857, 1713.4268]]], dtype=float32)

Okay, those look like pixel coordinates; but why are they all almost the same point? This also doesn't make sense.

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[-1.206582 , -0.8626694],
        [ 1.2490402, -0.9112338],
        [-1.8511834,  1.3299862],
        [ 2.2616546,  1.5442562]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, R=numpy.eye(3), P=new_K):

>>> cv2.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[265.85552573993016, 0.0, 320.7336534207575], [0.0, 266.30000369971407, 229.6288582732103], [0.0, 0.0, 1.0]]), 
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[1.2351158806052929, 0.0, 2268.4171643974255], [0.0, 1.2371808434650273, 1713.4398768258288], [0.0, 0.0, 1.0]])
)

array([[[2266.9248, 1712.3696],
        [2268.3188, 1713.4639],
        [2268.4397, 1713.3184],
        [2268.3857, 1713.4268]]], dtype=float32)

Okay, those look like pixel coordinates; but why are they all almost the same point? This also doesn't make sense.

Using cv2.fisheye.undistortPoints() gave different results, but they still look wrong.

array([[[  0.94239926,   0.67358345],
        [  0.13487473,  -0.09684527],
        [ 29.207176  , -22.761654  ],
        [  1.4594778 ,   1.1426234 ]]], dtype=float32)


array([[[2240.71  , 1693.6029],
        [2151.2527, 1725.2854],
        [2284.9624, 1625.003 ],
        [2151.2527, 1625.003 ]]], dtype=float32)

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.undistortPoints(
cv2.fisheye.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[-1.206582 array([[[  0.94239926,   0.67358345],
        [  0.13487473,  -0.09684527],
        [ 29.207176  , -0.8626694],
-22.761654  ],
        [ 1.2490402, -0.9112338],
        [-1.8511834,  1.3299862],
        [ 2.2616546,  1.5442562]]],  1.4594778 ,   1.1426234 ]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, R=numpy.eye(3), P=new_K):

>>> cv2.undistortPoints(
cv2.fisheye.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[265.85552573993016, np.array([[1076.7148792467171, 0.0, 320.7336534207575], 1298.9712963540678], [0.0, 266.30000369971407, 229.6288582732103], 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]), 
1.0]]),
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[1.2351158806052929, P=np.array([[416.0971612201596, 0.0, 2268.4171643974255], 1304.304969960433], [0.0, 1.2371808434650273, 1713.4398768258288], 416.79282483962464, 927.3730022048695], [0.0, 0.0, 1.0]])
)

array([[[2266.9248, 1712.3696],
        [2268.3188, 1713.4639],
        [2268.4397, 1713.3184],
        [2268.3857, 1713.4268]]], array([[[ -8029.981 ,  -5755.497 ],
        [  9563.489 ,  -5012.9556],
        [ 27344.436 , -19400.076 ],
        [-39704.24  , -31231.846 ]]], dtype=float32)

Okay, those look more like pixel coordinates; coordinates, but why are they all almost the same point?those still make no sense. This also doesn't make sense.

Using cv2.fisheye.undistortPoints() gave different results, but they still look wrong.

array([[[  0.94239926,   0.67358345],
        [  0.13487473,  -0.09684527],
        [ 29.207176  , -22.761654  ],
        [  1.4594778 ,   1.1426234 ]]], dtype=float32)


array([[[2240.71  , 1693.6029],
        [2151.2527, 1725.2854],
        [2284.9624, 1625.003 ],
        [2151.2527, 1625.003 ]]], dtype=float32)

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.

Not the most advanced OpenCV user/math-skilled individual, so please bear with me.

I've been following this short tutorial in an effort to calibrate a fisheye lens in OpenCV. So far, everything seems to be working as the tutorial prescribes: I was able to obtain working camera and distance coefficients, and successfully undistort images (i.e. running it through the provided code produces images that appear correct.) Following the second part of the tutorial, I've also been able to adjust the balance.

However, my application is that I want to undistort certain points (namely contours and the centers of bounding boxes) rather than entire images for performance reasons. As such, I thought I'd use cv2.undistortPoints(). My understanding is this should produce "ideal point coordinates", i.e. pixel coordinates corrected for the lens distortion. However, this doesn't appear be working as I expected.

Since the tutorial gives a K and a D matrix at the end, I figured I'd just plug those into undistortPoints.

>>> cv2.fisheye.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.016205134569390902], [-0.02434305021164351], [0.024555436941429715], [-0.008590717479362648]])
)

array([[[  0.94239926,   0.67358345],
        [  0.13487473,  -0.09684527],
        [ 29.207176  , -22.761654  ],
        [  1.4594778 ,   1.1426234 ]]], dtype=float32)

Those sure aren't pixel coordinates. I thought that maybe they were normalized points, with the bounds of the image being -1 and 1, but these values still don't make sense even within that context.

I also attempted to plug in the values obtained from the second part of the tutorial using balance=1.0. If you're looking at the tutorial, this corresponds to cv2.undistortPoints(my_test_points, scaled_K, dist_coefficients, R=numpy.eye(3), P=new_K):

>>> cv2.fisheye.undistortPoints(
  np.asarray([[[0, 0], [2592, 0], [0, 1944], [2592, 1944]]], dtype=np.float32),
  np.array([[1076.7148792467171, 0.0, 1298.9712963540678], [0.0, 1078.515014983842, 929.9968760065017], [0.0, 0.0, 1.0]]),
  np.array([[-0.019215744220979738], [-0.022168383678588813], [0.018999857407644722], [-0.003693599912847022]]), 
  R=np.eye(3), 
  P=np.array([[416.0971612201596, 0.0, 1304.304969960433], [0.0, 416.79282483962464, 927.3730022048695], [0.0, 0.0, 1.0]])
)

array([[[ -8029.981 ,  -5755.497 ],
        [  9563.489 ,  -5012.9556],
        [ 27344.436 , -19400.076 ],
        [-39704.24  , -31231.846 ]]], dtype=float32)

Okay, those look more like pixel coordinates, but those still make no sense.

At this point, I'm really not sure what to do. I've been struggling with this for quite some time now, so any and all help is truly appreciate. If you need the images, matrices, or anything else from me, I'm happy to provide it.

My camera is a 175° FOV RPi Camera (K) mounted on a Raspberry Pi, with the resolution at the maximum 2592×1944 for the purposes this question. I'm using OpenCV 3.4.4 with Python 3.