Problem when transporting points to a bigger image
Hi,
I'm developing a document scanner library from Android and I have almost everthing sorted but there's one thing that I can't get an appropiate solution. The library detects, crops and fixes the perspective for the document it finds, on the preview the contour is found perfectly adjusted to the document so it stores this contour and takes a full resolution picture, then multiplies (Core.multiply) the found set of points to crop the full-sized image and in that image I don't get the same adjustment that in the preview.
Here is an example of the preview (the document is lightly highlighted with a blue rectangle):
and the full-sized image (that has some annoying margin):
Has someone any idea what can be happening?
Thanks a lot.
EDIT:
The code I use to find the contour:
Imgproc.GaussianBlur(mat, mat, new Size(5.d, 5.d), 0);
Imgproc.Canny(mat, mat, 40, 180);
Imgproc.findContours(imageMat, points, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
for (int i = 0; i < points.size(); ++i) {
MatOfInt convexedPoints = new MatOfInt();
Imgproc.convexHull(points.get(i), convexedPoints);
points.set(i, convertIndexesToPoints(points.get(i), convexedPoints));
}
// sort points by size
MatOfPoint approxPoint = null;
for (MatOfPoint p : points) {
MatOfPoint2f c = new MatOfPoint2f(p.toArray());
double peri = Imgproc.arcLength(c, true);
MatOfPoint2f approx = new MatOfPoint2f();
Imgproc.approxPolyDP(c, approx, 0.1 * peri, true);
if (approx.toArray().length == 4 && Imgproc.contourArea(p) >= (imageArea * 0.25)) {
// this is the contour
}
}
And the code to convert the detected contour to the full-sized image:
double scaleWidth = imgMat.width() / previewSizeWidth;
double scaleHeight = imgMat.height() / previewSizeHeight;
MatOfPoint scaled = new MatOfPoint();
Core.multiply(detectedPoint, new Scalar(scaleWidth, scaleHeight), scaled);
// from scaled, sort points to get topLeft, topRight, ...
// calculate new width and height
double widthA = Math.sqrt(
Math.pow(bottomRight.x - bottomLeft.x, 2) +
Math.pow(bottomRight.y - bottomLeft.y, 2)
);
double widthB = Math.sqrt(
Math.pow(topRight.x - topLeft.x, 2) +
Math.pow(topRight.y - topLeft.y, 2)
);
double width = Math.max(widthA, widthB);
double heightA = Math.sqrt(
Math.pow(topRight.x - bottomRight.x, 2) +
Math.pow(topRight.y - bottomRight.y, 2)
);
double heightB = Math.sqrt(
Math.pow(topLeft.x - bottomLeft.x, 2) +
Math.pow(topLeft.y - bottomLeft.y, 2)
);
double height = Math.max(heightA, heightB);
// prepare mats for perspective and transform
MatOfPoint2f src = new MatOfPoint2f(
topLeft,
topRight,
bottomRight,
bottomLeft
);
MatOfPoint2f dest = new MatOfPoint2f(
new Point(0, 0),
new Point(width - 1, 0),
new Point(width - 1, height - 1),
new Point(0, height - 1)
);
// prepare mats for perspective and transform
MatOfPoint2f src = new MatOfPoint2f(
topLeft,
topRight,
bottomRight,
bottomLeft
);
MatOfPoint2f dest = new MatOfPoint2f(
new Point(0, 0),
new Point(width - 1, 0),
new Point(width - 1, height - 1),
new Point(0, height - 1)
);
// apply perspective and transform
Mat perspective = Imgproc.getPerspectiveTransform(src, dest);
Mat finalImage = new Mat((int) height, (int) width, CvType.CV_8UC1);
Imgproc.warpPerspective(image, finalImage, perspective, new Size(width, height));
Sorry for not attaching code earlier.
Try taking low and high resolution pictures of a chessboard test pattern. Do they actually line up? It's possible the low resolution image has some cropping, not just downscaling.
Thanks, will try that.
The test with a chessboard gives rectangular shapes instead of square shapes and same margin problem. I've updated the question with those images to see if anyone can help me out. Thanks!
Actually, without a single line of code it is quite guessing where your correction is going wrong. I have done exactly the same for page cropping and never met this issue once you indeed find the contour.
Sorry Steven, I've added some code to see if someone has some insight. Thanks!
You say it has a preview, then it takes the high resolution image. Save both, before the warp perspective, and make sure they actually line up.
Both images line up perfectly, and even the warp perspective on the preview gives a perfectly cropped and square image. I guess it can be something about the scale but I really don't know what. Thanks