using StereoBM in java on Android -> some results
Hello is someone could give me an example to show how to use org.opencv.calib3d.StereoBM in Java (on Android in my case) ? I only found examples in C++ and Python on the web...
EDIT2 : I tried to calibrate one camera before trying to calibrate both. So I wrote the following java code :
init variables :
imageCorners = new MatOfPoint2f();
savedImage = new Mat();
imagePoints = new ArrayList<>();
objectPoints = new ArrayList<>();
intrinsic = new Mat(3, 3, CvType.CV_32FC1);
distCoeffs = new Mat();
boardSize = new Size(numCornersHor, numCornersVer);
obj = new MatOfPoint3f();
for (int j = 0; j < numSquares; j++)
obj.push_back(new MatOfPoint3f(new Point3(j / numCornersHor, j % numCornersVer, 0.0f)));
In the main loop :
if (!isCalibrated) {
if (touch_screen_down) {
Mat grayImage = new Mat(rgba.size(), CvType.CV_8UC1);
Imgproc.cvtColor(rgba, grayImage, Imgproc.COLOR_BGR2GRAY);
boolean found = Calib3d.findChessboardCorners(
grayImage, boardSize, imageCorners,
Calib3d.CALIB_CB_ADAPTIVE_THRESH + Calib3d.CALIB_CB_NORMALIZE_IMAGE + Calib3d.CALIB_CB_FAST_CHECK);
if (found && successes <= boardsNumber) {
// optimization
TermCriteria term = new TermCriteria(TermCriteria.EPS | TermCriteria.MAX_ITER, 30, 0.001);
Imgproc.cornerSubPix(grayImage, imageCorners,
new Size(11, 11), new Size(-1, -1), term);
// save the current frame for further elaborations
grayImage.copyTo(savedImage);
// show the chessboard inner corners on screen
Calib3d.drawChessboardCorners(rgba, boardSize,
imageCorners, found);
imagePoints.add(imageCorners);
imageCorners = new MatOfPoint2f();
objectPoints.add(obj);
successes++;
}
if (successes == boardsNumber) {
List<Mat> rvecs = new ArrayList<>();
List<Mat> tvecs = new ArrayList<>();
intrinsic.put(0, 0, 1);
intrinsic.put(1, 1, 1);
calib_error = Calib3d.calibrateCamera(objectPoints, imagePoints, savedImage.size(), intrinsic, distCoeffs, rvecs, tvecs);
isCalibrated = true;
}
}
}
else { // is already calibrated so undistord the picture
Mat undistored = new Mat();
Imgproc.undistort(rgba, undistored, intrinsic, distCoeffs);
undistored.copyTo(rgba);
}
BUT the calibration error (calib_error) is bad when I run this code with 20 pictures of a chessboard : values are between 150 and 200. I read that the calibration error should be less than 0.5 !! And when calibration is done, news pictures from my camera are very distorded... What is the problem ?
EDIT1 : I wrote the following java code :
// load left and right images
Mat rgba_left = Utils.loadResource(
MainActivity.this,
R.drawable.trampoline3d_gauche,
Imgcodecs.CV_LOAD_IMAGE_COLOR);
Mat rgba_right = Utils.loadResource(
MainActivity.this,
R.drawable.trampoline3d_droite,
Imgcodecs.CV_LOAD_IMAGE_COLOR);
// create matrix with 1 channel
Mat mleft= new Mat(rgba_left.size(), CvType.CV_8UC1);
Mat mright= new Mat(rgba_right.size(), CvType.CV_8UC1);
Mat mdisparity= new Mat(rgba_left.size(), CvType.CV_8UC1);
// convert images to gray scale
Imgproc.cvtColor(rgba_left, mleft, Imgproc.COLOR_BGR2GRAY);
Imgproc.cvtColor(rgba_right, mright, Imgproc.COLOR_BGR2GRAY);
StereoBM stereo = StereoBM.create(16*5, 21);
stereo.setPreFilterSize(5);
stereo.setPreFilterCap(61);
stereo.setMinDisparity(-10);
stereo.setNumDisparities(16*5);
stereo.setTextureThreshold(300);
stereo.setUniquenessRatio(5);
stereo.setSpeckleWindowSize(0);
stereo.setSpeckleRange(8);
stereo.compute(mleft, mright, mdisparity);
// normalize from float to bytes
Core.normalize(mdisparity,rgba_right,0,255,Core.NORM_MINMAX,CvType.CV_8UC1);
// display the result
Imgproc.cvtColor(rgba_right, rgba_left, Imgproc.COLOR_GRAY2BGRA, 4);
Imgproc.resize(rgba_left, rgba, rgba.size());
but I obtain poor result ...
left image :
right image :
result :
do you have a stereo camera there ?
yes I have a stereo camera there
Only rectified image input will result in valid/useful stereo matching disparity results.
These are unrectified, raw, left and right images.
The cameras have slightly different pointing and rotation, and possibly slightly different focal length and distortion.
Cameras must be calibrated: intrinsic focal length and distortion of each camera is determined (mono calibration), and the rotation/translation of the second camera relative to the first is determined (stereo calibration).
The raw images need to be rectified into epipolar images, where focal lengths are idealized and distortion free. In epipolar images, real world features are visible on the same row line in both the left and right image.
Then run the stereo disparity calculation on the rectified images.
ok thanks for the answer. I was not sure of the need for calibration. Do you know a good (and easy) tutorial please ?
I don't think there is a tutorial for Java, but there are OpenCV documentation tutorials for Python and (partially) for C++. In any case, the theory and general method of solution using OpenCV is the same in any language. The Python tutorial:
Camera Calibration and 3D Reconstruction
The C++ tutorial, which doesn't go as far as calculating disparity, see:
Camera Calibration and 3D Reconstruction (calib3d module)
You may find some useful Java examples by looking at Boofcv and "Related links", but I can't comment on their currency or quality.
I worte java code to calibrate a camera but the result is very bad. See EDIT2. I don't understand why.