Revision history [back]

I'm running into a spot of unexpected behaviour from solvePnPRansac, one would expect if you calculate the rotation and translation vector using a dataset from 3D and 2D datapoints which were matched before that you'd be able to extract the 2D coordinates matching with the 3D ones again, sadly this doesn't seem to be the case when running the following minimal code:

import cv2
import numpy as np

f = 24
sx = 23.2
sy = 15.4
width = 6016
height = 4000
fx = width * f / sx
fy = height * f / sy
cx = width / 2
cy = height / 2

mtx = np.array([[fx, 0, cx], [0, fy, cy], [0, 0, 1]], dtype=np.float32)
objp = np.array([[336.2260,  638.2822, -131.7110],  [292.7631,  637.8978, -136.6205],  [292.4433,  518.9155, -136.4933],  [500.7705,  517.8542, -137.2822],  [430.0151,  610.0834, -161.7526],  [364.9480,  610.3971, -161.5365],  [502.4865,  635.9937, -137.2524],  [364.1737,  545.6596, -161.4099], [429.0482,  545.2378, -161.5822],  [396.7234,  577.5511, -131.6136]], dtype=np.float32)
imp = np.array([[2089, 1123],[1507, 1153],[1400, 2863],[4566, 2875],[3421, 1645],[2539, 1621],[4483, 1171],[2539, 2521],[3439, 2521],[2983, 1981]], dtype=np.float32)
dist = np.array([])

print(mtx)
print(objp)
print(imp)

_, rvecs, tvecs, inliers = cv2.solvePnPRansac(objp, imp, mtx, dist)

print(rvecs)
print(tvecs)
print(inliers)

imgpts, jac = cv2.projectPoints(objp, rvecs, tvecs, mtx, dist)

print(imgpts)

To be precise the output is the following:

Camera Matrix:
    [[  6.22344824e+03   0.00000000e+00   3.00800000e+03]
     [  0.00000000e+00   6.23376611e+03   2.00000000e+03]
     [  0.00000000e+00   0.00000000e+00   1.00000000e+00]]

3D Points
    [[ 336.22601318  638.28222656 -131.71099854]
     [ 292.76309204  637.89782715 -136.62049866]
     [ 292.44329834  518.91552734 -136.49330139]
     [ 500.77050781  517.85418701 -137.28219604]
     [ 430.0151062   610.08337402 -161.75259399]
     [ 364.94799805  610.39709473 -161.53649902]
     [ 502.48651123  635.99371338 -137.25239563]
     [ 364.17370605  545.65960693 -161.40989685]
     [ 429.04818726  545.23779297 -161.5821991 ]
     [ 396.72338867  577.55108643 -131.61360168]]

2D Points:
    [[ 2089.  1123.]
     [ 1507.  1153.]
     [ 1400.  2863.]
     [ 4566.  2875.]
     [ 3421.  1645.]
     [ 2539.  1621.]
     [ 4483.  1171.]
     [ 2539.  2521.]
     [ 3439.  2521.]
     [ 2983.  1981.]]

Rotation Vector:
    [[-0.9784294 ]
     [ 0.51393317]
     [ 0.7867821 ]]

Translation Vector:
    [[ -42.0280413 ]
     [ -78.03000936]
     [ 156.70873622]]

Inliers:
    None

Calculated 2D points:
    [[[ 7373.86523438   284.01651001]]

     [[ 8017.08056641   448.40975952]]

     [[ 7350.13916016   815.67822266]]

     [[ 4649.390625     296.13134766]]

     [[ 5951.33935547   517.70758057]]

     [[ 6725.28710938   664.42297363]]

     [[ 5423.21484375    71.22388458]]

     [[ 6334.44677734   848.40484619]]

     [[ 5543.67333984   675.48657227]]

     [[ 6205.56494141   298.80780029]]]

So either I'm doing something horribly wrong, or this is a significant portion of unexpected behaviour. I also run into this issue when using the matlab interface (mexopencv).