# Revision history [back]

Ok guys, first thanks a lot for your time and your help. Now lets go to the next step. I have computed the intrinsic, extrinsic coeffs and the rotation, translation matrices from stereoCalibrate(). Moreover I have extracted the R1,R2,P1,P2,Q matrices from the stereoRectify() (not sure if are needed):

Intrinsic1 (RGB): [267.3006709156614, 0, 210.9717308150853; 0, 267.9269486090523, 238.3792018571582; 0, 0, 1]

DistCoeffs1 (RGB): [-0.3360981423986475; 0.1561719100193124; -0.001470386476953027; -0.00138927595752683; -0.04271069396015477]

Intrinsic2 (IR): [88.31594825139115, 0, 79.36630195734749; 0, 88.46693314154564, 59.89706179624243; 0, 0, 1]

DistCoeffs2 (IR): [-0.07188957839421595; -0.02849295766534026; 0.0008792720966274496; -0.001737461395693609; 0.02205361597190071]

Rotation matrix (R): [0.9999707965902719, 0.003634151126819314, -0.006723013625189893; -0.003503519270297222, 0.9998068004051194, 0.01934133440071604; 0.006792004073887695, -0.01931721536001252, 0.9997903349559822]

Translation matrix (T):
[-52.44976949785277;
97.05478134052377;
11.74009367842509]

Essensial matrix:
[]

Fundamental matrix:
[]

R1:
[0.8793202765214395, 0.4752250730842713, 0.03093511290278861;
-0.4750938841337006, 0.870882191746457, 0.1258968202840782;
0.03288848669387377, -0.125400709770466, 0.9915608954734199]

R2:
[0.8808136599204212, 0.4726508706224962, 0.02772094871012055; -0.4727614983120196, 0.8748134507391853, 0.1058205656740386; 0.02576552369673482, -0.1063135969930264, 0.9939987710671666]

P1: [63.18380226578106, 0, 230.5816181898117, 0; 0, 63.18380226578106, 209.8756529316306, 0; 0, 0, 1, 0]

P2: [63.18380226578106, 0, 230.5816181898117, 0; 0, 63.18380226578106, 209.8756529316306, 7009.826046898012; 0, 0, 1, 0]

Q: [1, 0, 0, -230.5816181898117; 0, 1, 0, -209.8756529316306; 0, 0, 0, 63.18380226578106; 0, 0, -0.009013604880215414, 0]

Now before I try to align the RGB with the depth, I would like to do that with the IR image as I am showing in my initial post. However, I am quite confused how to do this. According to the documentation initUndistortRectifyMap() should do that but my output is really strange as you can see:

Ok guys, first thanks a lot for your time and your help. Now lets go to the next step. I have computed the intrinsic, extrinsic coeffs and the rotation, translation matrices from stereoCalibrate(). Moreover I have extracted the R1,R2,P1,P2,Q matrices from the stereoRectify() (not sure if are needed):

Intrinsic1 (RGB):
[267.3006709156614, 0, 210.9717308150853;
0, 267.9269486090523, 238.3792018571582;
0, 0, 1] 1]

DistCoeffs1 (RGB):
[-0.3360981423986475;
0.1561719100193124;
-0.001470386476953027;
-0.00138927595752683;
-0.04271069396015477] -0.04271069396015477]

Intrinsic2 (IR):
[88.31594825139115, 0, 79.36630195734749;
0, 88.46693314154564, 59.89706179624243;
0, 0, 1] 1]

DistCoeffs2 (IR):
[-0.07188957839421595;
-0.02849295766534026;
0.0008792720966274496;
-0.001737461395693609;
0.02205361597190071] 0.02205361597190071]

Rotation matrix (R):
[0.9999707965902719, 0.003634151126819314, -0.006723013625189893;
-0.003503519270297222, 0.9998068004051194, 0.01934133440071604;
0.006792004073887695, -0.01931721536001252, 0.9997903349559822] 0.9997903349559822]

Translation matrix (T):
[-52.44976949785277;
97.05478134052377;
11.74009367842509]                                                                                                                         Essensial matrix:
[]                                                                                                                                          Fundamental matrix:
[]                                                                                                                                          R1:
[0.8793202765214395, 0.4752250730842713, 0.03093511290278861;
-0.4750938841337006, 0.870882191746457, 0.1258968202840782;
0.03288848669387377, -0.125400709770466, 0.9915608954734199]                                                                               R2:
[0.8808136599204212, 0.4726508706224962, 0.02772094871012055;
-0.4727614983120196, 0.8748134507391853, 0.1058205656740386;
0.02576552369673482, -0.1063135969930264, 0.9939987710671666] 0.9939987710671666]

P1:
[63.18380226578106, 0, 230.5816181898117, 0;
0, 63.18380226578106, 209.8756529316306, 0;
0, 0, 1, 0] 0]

P2:
[63.18380226578106, 0, 230.5816181898117, 0;
0, 63.18380226578106, 209.8756529316306, 7009.826046898012;
0, 0, 1, 0] 0]

Q:
[1, 0, 0, -230.5816181898117;
0, 1, 0, -209.8756529316306;
0, 0, 0, 63.18380226578106;
0, 0, -0.009013604880215414, 0]0]


Now before I try to align the RGB with the depth, I would like to do that with the IR image as I am showing in my initial post. However, I am quite confused how to do this. According to the documentation initUndistortRectifyMap() should do that but my output is really strange as you can see:

while if I use the common undistort() function I am getting a better result:

distorted:

Undistorted:

my other question here is can I get my undistorted image without being zoomed in? and in the surounding pixels being black. In general how do I align the rgb with the ir image and then apply the distortion again back (I guess the same procedure will be followed for the depth image).

Ok guys, first thanks a lot ok I tried to obtain some better intrinsic and dist coeffs for your time the rgb camera. Then using the undistortion() function I undistorted the images and your help. Now lets go finally multiplying the dist coeffs with -1 and reapplying the undistortion() function I reapplied the distortion. However, you will notice that the result is wrong. Why this, is it due to the next step. I have computed the intrinsic, extrinsic coeffs and the rotation, translation matrices from stereoCalibrate(). Moreover I have extracted the R1,R2,P1,P2,Q matrices from the stereoRectify() (not sure if are needed):

Intrinsic1 (RGB):
[267.3006709156614, 0, 210.9717308150853;
0, 267.9269486090523, 238.3792018571582;
0, 0, 1]

DistCoeffs1 (RGB):
[-0.3360981423986475;
0.1561719100193124;
-0.001470386476953027;
-0.00138927595752683;
-0.04271069396015477]

Intrinsic2 (IR):
[88.31594825139115, 0, 79.36630195734749;
0, 88.46693314154564, 59.89706179624243;
0, 0, 1]

DistCoeffs2 (IR):
[-0.07188957839421595;
-0.02849295766534026;
0.0008792720966274496;
-0.001737461395693609;
0.02205361597190071]

Rotation not good calibration matrix (R):
[0.9999707965902719, 0.003634151126819314, -0.006723013625189893;
-0.003503519270297222, 0.9998068004051194, 0.01934133440071604;
0.006792004073887695, -0.01931721536001252, 0.9997903349559822]

Translation matrix (T):
[-52.44976949785277;
97.05478134052377;
11.74009367842509]

Essensial matrix:
[]

Fundamental matrix:
[]

R1:
[0.8793202765214395, 0.4752250730842713, 0.03093511290278861;
-0.4750938841337006, 0.870882191746457, 0.1258968202840782;
0.03288848669387377, -0.125400709770466, 0.9915608954734199]

R2:
[0.8808136599204212, 0.4726508706224962, 0.02772094871012055;
-0.4727614983120196, 0.8748134507391853, 0.1058205656740386;
0.02576552369673482, -0.1063135969930264, 0.9939987710671666]

P1:
[63.18380226578106, 0, 230.5816181898117, 0;
0, 63.18380226578106, 209.8756529316306, 0;
0, 0, 1, 0]

P2:
[63.18380226578106, 0, 230.5816181898117, 0;
0, 63.18380226578106, 209.8756529316306, 7009.826046898012;
0, 0, 1, 0]

Q:
[1, 0, 0, -230.5816181898117;
0, 1, 0, -209.8756529316306;
0, 0, 0, 63.18380226578106;
0, 0, -0.009013604880215414, 0]


Now before I try to align the RGB with the depth, I would like to do that with the IR image as I am showing in my initial post. However, I am quite confused how to do this. According to the documentation initUndistortRectifyMap() should do that but my output is really strange as you can see:or something else?

Original image:

while if I use the common undistort() function I am getting a better result:

distorted:Undistorted image (how I do not zoom in here??????????)

Undistorted:Redistorted image:

my other question here is can I get my undistorted Overlayed redistorted image without being zoomed in? and in to original (notice the surounding pixels being black. In general how do I align the rgb with the ir image and then apply the distortion again back (I guess the same procedure will be followed for the depth image).bad allignement):