How to retrieve Object Points of CharUco Board

edit

Hello,

When detecting corners for a charuco board, I follow:

```

    image = <IMAGE OF BOARD>

    dictionary = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_5X5_1000)
    board = cv2.aruco.CharucoBoard_create(
        num_squares_x, num_squares_y, square_length, marker_length, dictionary)
    corners, ids, _ = cv2.aruco.detectMarkers(img, dictionary)

    if ids is not None and ids.size > 0:
          num_corners, charuco_corners, charuco_ids = cv2.aruco.interpolateCornersCharuco(
                corners, ids, img, board)

```

After getting a calibration, its easy to find corresponding object_points and image_points to pass into something like solvePnP when using the ArUco corners. I.e.

```

    obj_pts, img_pts = cv2.aruco.getBoardObjectAndImagePoints(
          board,
          corners,
          ids
    )

```

However I am having a hard time finding the corresponding charuco_object_pts that match with charuco_corners. There doesn't seem to be a function in cv2.aruco that recovers this for you. Is there a standard convention that specifies what charuco_object_pts are.

I know for normal checkerboards such object points could be recovered like ```

obj_pattern = np.zeros((num_squares_y * num_squares_x, 3), np.float32)
obj_pattern[:, :2] = np.mgrid[0:num_squares_y, 0:num_squares_x].T.reshape(-1, 2)

```
But this does not work for chaUrco Board from my experiments. Any help would be greatly appreciated!