Homography holds for the following cases:
- planar scene
- general world but rotational motion (e.g. panorama)
If you want to align the left infrared image to the color image, you can:
- align the color image with the depth stream
- the left IR and the color images are now aligned since the left IR is aligned with the depth stream by design
Another possibility:
- compute the [uv] map that aligns the depth image to the color image
- apply the [uv] map to the left IR image