Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

The maths should be the following.

Given the color K_c and depth K_d intrinsic parameters:

eq1

Given the homogeneous transformation between the color and depth frames:

eq2

The previous matrix allows transforming a 3D point expressed in one frame to another frame:

eq3

To get the corresponding RGB color for a given 3D point:


(Skip this as you already have the [X,Y,Z] coordinates. Sometimes, you only have the depth map.)

Get the full 3D coordinate from the depth value Z_d (using the depth intrinsic parameters) for a given [u,v] coordinate in the depth map (here without taking into account the distortion):

eq4

eq5


Transform the 3D point expressed in the depth frame to the color frame:

eq6

Project the 3D point expressed in the color frame into the image plane to get the corresponding [u_c, v_c] coordinate:

eq7


Practically, you should be able to achieve this using projectPoints(). More details about the camera frame here.