From the OpenCV documentation I understand that pixel is the unit used to process the distortion effects induced by imperfect objective. As an example in the camera matrix the focal lens must be given in pixel. On the other hand, I need to use the distortion coefficients k1,k2,p1,p2,k3 provided by the commercial software iWitness for further image processing by means OpenCV. The unit used in iWitness is mm, so that typically k1 is about 1e-04. This mean that Dx = k1r^3, for r=10 mm is 0.1 mm.
I tried to rescale the coefficients to be applied in distance in pixel, that is
k1_OpenCv = k1_iWitness*px_dimension^2, but I get very small k1 value. As an example for Nikon D800, px_dimension = 4.9e-03 mm, so that k1_OpenCV = 2.4e-09 that is much smaller than the values reported by many users, like 0.1.
Looking at the source of cv::initUndistortRectifyMap it seems that x,y are, in a quite complicated manne,r normalized to size.width size.height respectively. Probably this is the key of the problem, but how to do that correctly?
How to transform (k1,k2,p1,p2,k3)_iWitness in (k1,k2,p1,p2,k3)_OpenCV?
Thanks for any help.