Ask Your Question
2

Strange epipolar lines and 3d reconstruction [OpenCV for Java]

asked 2014-12-30 17:23:52 -0600

respectum gravatar image

Hi,
I'm working on my 3d reconstruction project and I'm having problems with obtaining proper epipolar lines.
Key points
Epilines
My main concern is that they not intersect keypoints.
I'm using SURF feature detector (java_3d_reconstruction\src\KeyPairsDetector.java)

private void detectKeyPoints(MatOfKeyPoint keyPoints1, MatOfKeyPoint keyPoints2) {
    FeatureDetector surfDetector = FeatureDetector.create(FeatureDetector.SURF);
    surfDetector.detect(getImg1(), keyPoints1);
    surfDetector.detect(getImg2(), keyPoints2);
}

than FLANNBASED matcher (java_3d_reconstruction\src\KeyPairsDetector.java)

DescriptorMatcher descriptorMatcher = DescriptorMatcher.create(DescriptorMatcher.FLANNBASED);
        ArrayList<MatOfDMatch> matches = new ArrayList<MatOfDMatch>();
        descriptorMatcher.knnMatch(descriptors1, descriptors2, matches, 1);

and I'm filtering good matches using mask obtained by findHomography with RANSAC (java_3d_reconstruction\src\KeyPairsDetector.java)

Mat mask = new Mat();
        Calib3d.findHomography(srcPoints, dstPoints, Calib3d.RANSAC, 5, mask);

        List<MatOfDMatch> goodMatches = getInliers(mask, matches);

Then I'm calculating fundamental matrix using 8 point algorithm (java_3d_reconstruction\src\EpipolarGeometry.java):

private void computeFundamentalMatrix() {
        fundamentalMatrix  = Calib3d.findFundamentalMat(srcPoints, dstPoints, Calib3d.FM_8POINT, 0.0, 0.0);
    }

computing epipolar lines (java_3d_reconstruction\src\EpipolarGeometry.java):

private void computeEpiLines() {
        epilinesSrc = new Mat();
        epilinesDst = new Mat();

        Calib3d.computeCorrespondEpilines(srcPoints, 1, fundamentalMatrix, epilinesDst);
        Calib3d.computeCorrespondEpilines(dstPoints, 2, fundamentalMatrix, epilinesSrc);
    }

and drawing them (java_3d_reconstruction\src\EpipolarGeometry.java):

public Mat drawEpiLines(Mat outImg) {

        int epiLinesCount = epilinesSrc.rows();

        double a, b, c;

        for (int line = 0; line < epiLinesCount; line++) {
            a = epilinesSrc.get(line, 0)[0];
            b = epilinesSrc.get(line, 0)[1];
            c = epilinesSrc.get(line, 0)[2];

            int x0 = 0;
            int y0 = (int) (-(c + a * x0) / b);
            int x1 = outImg.cols() / 2;
            int y1 = (int) (-(c + a * x1) / b);

            Point p1 = new Point(x0, y0);
            Point p2 = new Point(x1, y1);
            Scalar color = new Scalar(255, 255, 255);
            Core.line(outImg, p1, p2, color);

        }

        for (int line = 0; line < epiLinesCount; line++) {
            a = epilinesDst.get(line, 0)[0];
            b = epilinesDst.get(line, 0)[1];
            c = epilinesDst.get(line, 0)[2];

            int x0 = outImg.cols() / 2;
            int y0 = (int) (-(c + a * x0) / b);
            int x1 = outImg.cols();
            int y1 = (int) (-(c + a * x1) / b);

            Point p1 = new Point(x0, y0);
            Point p2 = new Point(x1, y1);
            Scalar color = new Scalar(255, 255, 255);
            Core.line(outImg, p1, p2, color);

        }
        return outImg;
    }

The main code:

public class Main {
    static {
        System.loadLibrary("opencv_java2410");
    }
    public static void main(String[] args) {
        System.loadLibrary("opencv_java2410");


        Mat img1 = Highgui.imread("images\\SylvainJpg\\S01.jpg", Highgui.CV_LOAD_IMAGE_GRAYSCALE);
        Mat img2 = Highgui.imread("images\\SylvainJpg\\S02.jpg", Highgui.CV_LOAD_IMAGE_GRAYSCALE);

        KeyPairsDetector keyPairsDetector = new KeyPairsDetector(img1,img2);
        EpipolarGeometry epipolarGeometry = new EpipolarGeometry(keyPairsDetector.srcSortedGoodPoints,
                keyPairsDetector.dstSortedGoodPoints);

        Mat matchedImg = keyPairsDetector.drawMatchesAndKeyPoints();
        epipolarGeometry.drawEpiLines(matchedImg);

        Highgui.imwrite("images\\SylvainJpg\\S00c.jpg", matchedImg);

    }
}

Whole code is available here GitHub repo

Could you please help me work out what is wrong here?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
3

answered 2015-01-06 10:30:12 -0600

respectum gravatar image

updated 2015-01-06 10:35:44 -0600

Ok, I've tracked it down, there were two problems:

  • Bad keypoints matching, although it looks quite good on this picture it is awfull on other simple images. I've used ratio test, symetry test, and than RANSAC mask from findFundamentalMatrix to choose good matches.
  • Bad epilines drawing

I've found bug in drawing method and for the second loop in public Mat drawEpiLines(Mat outImg) I've used

Point p1 = new Point(x0+outImg.cols()/2, y0);
Point p2 = new Point(x1+outImg.cols()/2, y1);

so I've added outImg.cols()/2 which transposed line for the second image. Results are here: image description

edit flag offensive delete link more

Comments

But that's still not right, is it? I mean it makes sense in the right image, but in the left image, the lines should converge to the right, as the second frame's camera center is further to the right.

themightyoarfish gravatar imagethemightyoarfish ( 2015-07-10 06:37:32 -0600 )edit

@themightyyoarfish make sure you understand how epipolar lines works like. When the movement is small lines do not converge into different sides.

respectum gravatar imagerespectum ( 2015-12-20 12:07:40 -0600 )edit

Question Tools

2 followers

Stats

Asked: 2014-12-30 17:23:52 -0600

Seen: 3,951 times

Last updated: Jan 06 '15