using StereoBM in java on Android -> some results

asked 2017-11-27 06:35:16 -0500

PATDESSE gravatar image

updated 2017-12-16 07:42:47 -0500

Hello is someone could give me an example to show how to use org.opencv.calib3d.StereoBM in Java (on Android in my case) ? I only found examples in C++ and Python on the web...

EDIT2 : I tried to calibrate one camera before trying to calibrate both. So I wrote the following java code :

init variables :

        imageCorners = new MatOfPoint2f();
        savedImage = new Mat();
        imagePoints = new ArrayList<>();
        objectPoints = new ArrayList<>();
        intrinsic = new Mat(3, 3, CvType.CV_32FC1);
        distCoeffs = new Mat();
        boardSize = new Size(numCornersHor, numCornersVer);
        obj = new MatOfPoint3f();
        for (int j = 0; j < numSquares; j++)
            obj.push_back(new MatOfPoint3f(new Point3(j / numCornersHor, j % numCornersVer, 0.0f)));

In the main loop :

                if (!isCalibrated) {
                    if (touch_screen_down) {
                        Mat grayImage = new Mat(rgba.size(), CvType.CV_8UC1);
                        Imgproc.cvtColor(rgba, grayImage, Imgproc.COLOR_BGR2GRAY);

                        boolean found = Calib3d.findChessboardCorners(
                                grayImage, boardSize, imageCorners,
                                Calib3d.CALIB_CB_ADAPTIVE_THRESH + Calib3d.CALIB_CB_NORMALIZE_IMAGE + Calib3d.CALIB_CB_FAST_CHECK);

                        if (found && successes <= boardsNumber) {
                            // optimization
                            TermCriteria term = new TermCriteria(TermCriteria.EPS | TermCriteria.MAX_ITER, 30, 0.001);
                            Imgproc.cornerSubPix(grayImage, imageCorners,
                                    new Size(11, 11), new Size(-1, -1), term);

                            // save the current frame for further elaborations
                            grayImage.copyTo(savedImage);

                            // show the chessboard inner corners on screen
                            Calib3d.drawChessboardCorners(rgba, boardSize,
                                    imageCorners, found);

                            imagePoints.add(imageCorners);
                            imageCorners = new MatOfPoint2f();
                            objectPoints.add(obj);
                            successes++;
                        }

                        if (successes == boardsNumber) {
                            List<Mat> rvecs = new ArrayList<>();
                            List<Mat> tvecs = new ArrayList<>();
                            intrinsic.put(0, 0, 1);
                            intrinsic.put(1, 1, 1);

                            calib_error = Calib3d.calibrateCamera(objectPoints, imagePoints, savedImage.size(), intrinsic, distCoeffs, rvecs, tvecs);
                            isCalibrated = true;
                        }
                    }
                }
                else {   // is already calibrated so undistord the picture
                    Mat undistored = new Mat();
                    Imgproc.undistort(rgba, undistored, intrinsic, distCoeffs);
                    undistored.copyTo(rgba);
                }

BUT the calibration error (calib_error) is bad when I run this code with 20 pictures of a chessboard : values are between 150 and 200. I read that the calibration error should be less than 0.5 !! And when calibration is done, news pictures from my camera are very distorded... What is the problem ?

EDIT1 : I wrote the following java code :

                // load left and right images
                Mat rgba_left = Utils.loadResource(
                        MainActivity.this, 
                        R.drawable.trampoline3d_gauche,
                        Imgcodecs.CV_LOAD_IMAGE_COLOR);
                Mat rgba_right = Utils.loadResource(
                        MainActivity.this, 
                        R.drawable.trampoline3d_droite,
                        Imgcodecs.CV_LOAD_IMAGE_COLOR);

                // create matrix with 1 channel
                Mat mleft= new Mat(rgba_left.size(), CvType.CV_8UC1);
                Mat mright= new Mat(rgba_right.size(), CvType.CV_8UC1);
                Mat mdisparity= new Mat(rgba_left.size(), CvType.CV_8UC1);

                // convert images to gray scale
                Imgproc.cvtColor(rgba_left, mleft, Imgproc.COLOR_BGR2GRAY);
                Imgproc.cvtColor(rgba_right, mright, Imgproc.COLOR_BGR2GRAY);

                StereoBM stereo = StereoBM.create(16*5, 21);
                stereo.setPreFilterSize(5);
                stereo.setPreFilterCap(61);
                stereo.setMinDisparity(-10);
                stereo.setNumDisparities(16*5);
                stereo.setTextureThreshold(300);
                stereo.setUniquenessRatio(5);
                stereo.setSpeckleWindowSize(0);
                stereo.setSpeckleRange(8);

                stereo.compute(mleft, mright, mdisparity);

                // normalize from float to bytes
                Core.normalize(mdisparity,rgba_right,0,255,Core.NORM_MINMAX,CvType.CV_8UC1);

                // display the result
                Imgproc.cvtColor(rgba_right, rgba_left, Imgproc.COLOR_GRAY2BGRA, 4);
                Imgproc.resize(rgba_left, rgba, rgba.size());

but I obtain poor result ...

left image : image description

right image : image description

result : image description

edit retag flag offensive close merge delete

Comments

do you have a stereo camera there ?

berak gravatar imageberak ( 2017-11-27 06:48:42 -0500 )edit

yes I have a stereo camera there

PATDESSE gravatar imagePATDESSE ( 2017-11-28 10:10:03 -0500 )edit

Only rectified image input will result in valid/useful stereo matching disparity results.

These are unrectified, raw, left and right images.

The cameras have slightly different pointing and rotation, and possibly slightly different focal length and distortion.

Cameras must be calibrated: intrinsic focal length and distortion of each camera is determined (mono calibration), and the rotation/translation of the second camera relative to the first is determined (stereo calibration).

The raw images need to be rectified into epipolar images, where focal lengths are idealized and distortion free. In epipolar images, real world features are visible on the same row line in both the left and right image.

Then run the stereo disparity calculation on the rectified images.

opalmirror gravatar imageopalmirror ( 2017-12-08 15:19:38 -0500 )edit

ok thanks for the answer. I was not sure of the need for calibration. Do you know a good (and easy) tutorial please ?

PATDESSE gravatar imagePATDESSE ( 2017-12-10 09:30:58 -0500 )edit

I don't think there is a tutorial for Java, but there are OpenCV documentation tutorials for Python and (partially) for C++. In any case, the theory and general method of solution using OpenCV is the same in any language. The Python tutorial:

Camera Calibration and 3D Reconstruction

The C++ tutorial, which doesn't go as far as calculating disparity, see:

Camera Calibration and 3D Reconstruction (calib3d module)

You may find some useful Java examples by looking at Boofcv and "Related links", but I can't comment on their currency or quality.

opalmirror gravatar imageopalmirror ( 2017-12-11 13:08:19 -0500 )edit

I worte java code to calibrate a camera but the result is very bad. See EDIT2. I don't understand why.

PATDESSE gravatar imagePATDESSE ( 2017-12-16 07:44:17 -0500 )edit