Revision history [back]

If I understand you correctly, you want to map the Depth (or IR) frame over the RGB frame, using kinect SDK V2.

I don't really understand, why do you want to calibrate the sensor yourself and match the frames in OpenCV. The Kinect is very well calibrated, and the SDK provides you all the needed functions in the CoordinateMapper class (frame-to-frame, point-to-frame). It's also much faster (it uses directly the depth data and precomputed correction tables and it runs on the GPU).

These functions are present in OpenNI and libfreenect as well.

Note that the precise correction data (z_table) is included in the firmware of each Kinect - so better use this than your own correction data. If you want to make the registration yourself, take a look at the source code of libfreenect2 (libfreenect2.cpp for setting the camera parameters and reading the z_table and registration.cpp for the registration).

If I understand you correctly, you want to map the Depth (or IR) frame over the RGB frame, using kinect SDK V2.

I don't really understand, why do you want to calibrate the sensor yourself and match the frames in OpenCV. The Kinect is very well calibrated, and the SDK provides you all the needed functions in the CoordinateMapper class (frame-to-frame, point-to-frame). It's also much faster (it uses directly the depth data and precomputed correction tables and it runs on the GPU).

These functions are present in OpenNI and libfreenect libfreenect2 as well.

[Answer updated based on the comments below]

Note that the precise correction data (z_table) is parameters are included in the firmware of each Kinect - so better use this than your own correction data. If you want to make the registration yourself, take are using offline registration, you should still use these parameters.

You can extract these parameters using libfreenect2 (Freenect2Device::getIrCameraParams and getColorCameraParams).

Then you can use the Registration::distort and Registration::depth_to_color functions with these parameters (from the registration.cpp).

If you don't have a look at the source code of libfreenect2 (libfreenect2.cpp for setting the camera parameters and reading the z_table and Kinect, you might try to use the parameters I extracted from my sensor:

IR: fx=365.481 fy=365.481 cx=257.346 cy=210.347 k1=0.089026 k2=-0.271706 k3=0.0982151 p1=0 p2=365.481

Color: fx=1081.37 fy=1081.37 cx=959.5 cy=539.5 shift_d=863 shift_m=52 mx_x3y0=0.000449294 mx_x0y3=1.91656e-05 mx_x2y1=4.82909e-05 mx_x1y2=0.000353673 mx_x2y0=-2.44043e-05 mx_x0y2=-1.19426e-05 mx_x1y1=0.000988431 mx_x1y0=0.642474 mx_x0y1=0.00500649 mx_x0y0=0.142021 my_x3y0=4.42793e-06 my_x0y3=0.000724863 my_x2y1=0.000398557 my_x1y2=4.90383e-05 my_x2y0=0.000136024 my_x0y2=0.00107291 my_x1y1=-1.75465e-05 my_x1y0=-0.00554263 my_x0y1=0.641807 my_x0y0=0.0180811

You might try to copy the functions I mentioned earlier from registration.cpp for the registration).