# Revision history [back]

### OpenCV: Essential Matrix SVD decomp

Hi Folks,

I am trying to get camera motion vector based on OpenCV Optical Flow. I use C# wrapper for Unity of OpenCV 2.4.10, but it is just wrapper Here is the test case:

1. Calibrated my camera and have camera matrix K (3x3)
2. Use 2 100%-identical images framePrev and frameThis as optical flow frames (means no motion)
3. Selected features (2d points) from both images via
goodFeaturesToTrack (frameThis, pointsThis, iGFFTMax, 0.05, 20);
goodFeaturesToTrack (framePrev, pointsPrev, iGFFTMax, 0.05, 20);
so i have features pointsPrev and pointsThis

4. Use
calcOpticalFlowPyrLK (framePrev, frameThis, pointsPrev, pointsThis, status, err);
to verify flow for points, then I make sure analyzing status and err arrays, so my pointsPrev and pointsThis are identical pairs of points in image pixel coordinates

5. Select first 8 pairs from pointsPrev and poitsThis (simply trunc arrays), then get Fundamental Matrix:
F = Calib3d.findFundamentalMat(pointsPrev, pointsThis, Calib3d.FM_8POINT, 2, 0.99); 
. When points in all the pairs are identical (no motion) - it gives me 3x3 matrix with all zeros, I suggest that is correct (or?)

6. Then getting Essential Matrix based on E = K'.(t) * F * K according to HZ 9.12, I have one camera, so K' = K.
gemm (K.t (),F,1,null,0,tmpMat,Core.GEMM_3_T);
gemm (tmpMat,K,1,null,0,E,Core.GEMM_3_T);

when F = |0|, then E = |0| as well

7. Finally I apply SVD decomposition on E:

SVDecomp(E,W,U,VT);

8. Analyzing W, U, VT output matrices, I can observe these values:
W: 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000
U: -0.577, -0.408, -0.707, -0.577, -0.408, 0.707, 0.577, -0.816, 0.000
Vt: 1.000, 0.000, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 1.000

I suggest these values are strange, as according to books/manuals camera translation vector is U.col(2) is Vector3(-0.7071, 0.7071, 0) that is not correct.

Other observations, when I test for different image frames:

• U values are always between -1 and 1, that should not be translation, similar more on sine/cosine values (again, 0.7071 is sine of pie/4 or cosine of pie/4)
• Fundamental matrix outputs are radically different for different algoriths - 8POINTS, 7POINTS, RANSAC, LMEDS, even for pairs of corresponding poits (features)
• using dirrefent number of pairs of points (features) - say 5, 7, 8, 15, 40 - for the same algoriths also radically changes fundamental matrix output

That is the copy of my question on StackOverflow : http://stackoverflow.com/questions/30953989/opencv-essential-matrix-svd-decomp

Kind Regards, Eugene

### OpenCV: Essential Matrix SVD decomp

Hi Folks,

I am trying to get camera motion vector based on OpenCV Optical Flow. I use C# wrapper for Unity of OpenCV 2.4.10, but it is just wrapper Here is the test case:

1. Calibrated my camera and have camera matrix K (3x3)
2. Use 2 100%-identical images framePrev and frameThis as optical flow frames (means no motion)
3. Selected features (2d points) from both images via
goodFeaturesToTrack (frameThis, pointsThis, iGFFTMax, 0.05, 20);
goodFeaturesToTrack (framePrev, pointsPrev, iGFFTMax, 0.05, 20);
so i have features pointsPrev and pointsThis

4. Use
calcOpticalFlowPyrLK (framePrev, frameThis, pointsPrev, pointsThis, status, err);
to verify flow for points, then I make sure analyzing status and err arrays, so my pointsPrev and pointsThis are identical pairs of points in image pixel coordinates

5. Select first 8 pairs from pointsPrev and poitsThis (simply trunc arrays), then get Fundamental Matrix:
F = Calib3d.findFundamentalMat(pointsPrev, pointsThis, Calib3d.FM_8POINT, 2, 0.99); 
. When points in all the pairs are identical (no motion) - it gives me 3x3 matrix with all zeros, I suggest that is correct (or?)

6. Then getting Essential Matrix based on E = K'.(t) * F * K according to HZ 9.12, I have one camera, so K' = K.
gemm (K.t (),F,1,null,0,tmpMat,Core.GEMM_3_T);
gemm (tmpMat,K,1,null,0,E,Core.GEMM_3_T);

when F = |0|, then E = |0| as well

7. Finally I apply SVD decomposition on E:

SVDecomp(E,W,U,VT);

8. Analyzing W, U, VT output matrices, I can observe these values:
W: 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000
U: -0.577, -0.408, -0.707, -0.577, -0.408, 0.707, 0.577, -0.816, 0.000
Vt: 1.000, 0.000, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 1.000

I suggest these values are strange, as according to books/manuals camera translation vector is U.col(2) is Vector3(-0.7071, 0.7071, 0) that is not correct.

Other observations, when I test for different image frames:

• U values are always between -1 and 1, that should not be translation, similar more on sine/cosine values (again, 0.7071 is sine of pie/4 or cosine of pie/4)
• Fundamental matrix outputs are radically different for different algoriths - 8POINTS, 7POINTS, RANSAC, LMEDS, even for pairs of corresponding poits (features)
• using dirrefent number of pairs of points (features) - say 5, 7, 8, 15, 40 - for the same algoriths also radically changes fundamental matrix output

That is the copy of my question on StackOverflow : http://stackoverflow.com/questions/30953989/opencv-essential-matrix-svd-decomp

Kind Regards, Eugene

EDIT 1: Additional observations Then I tried to find the Fund matrix for these frame poits:


MatOfPoint2f p1 = new MatOfPoint2f(new Point(100,100),new Point(100,200),new Point(100,300),
new Point (200,100),new Point(200,200),new Point(200,300),
new Point(300,100),new Point(300,200),new Point(300,300));
MatOfPoint2f p2 = new MatOfPoint2f(new Point(80,80),new Point(80,200),new Point(80,320),
new Point (200,80),new Point(200,200),new Point(200,320),
new Point(320,80),new Point(320,200),new Point(320,320));


The points correspond to case when camera moves forward direction, all the features are center-symmetrical. When I use findFundamentalMat with 8POINT algorithm - The Fund matrix is


F = 0.00000000, 0.00010236, -0.02047281, -0.00010236, 0.00000000, 0.02047281, 0.02047281, -0.02047281, 0.00000000, 

But when I use RANSAC - the result is


F = 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000,