Ask Your Question
1

Problem when transporting points to a bigger image

asked 2017-03-21 15:03:38 -0600

khrizt gravatar image

updated 2017-03-22 10:01:27 -0600

Hi,

I'm developing a document scanner library from Android and I have almost everthing sorted but there's one thing that I can't get an appropiate solution. The library detects, crops and fixes the perspective for the document it finds, on the preview the contour is found perfectly adjusted to the document so it stores this contour and takes a full resolution picture, then multiplies (Core.multiply) the found set of points to crop the full-sized image and in that image I don't get the same adjustment that in the preview.

Here is an example of the preview (the document is lightly highlighted with a blue rectangle):

image description

and the full-sized image (that has some annoying margin):

image description

Has someone any idea what can be happening?

Thanks a lot.

EDIT:

The code I use to find the contour:

Imgproc.GaussianBlur(mat, mat, new Size(5.d, 5.d), 0);
Imgproc.Canny(mat, mat, 40, 180);
Imgproc.findContours(imageMat, points, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

for (int i = 0; i < points.size(); ++i) {
    MatOfInt convexedPoints = new MatOfInt();
    Imgproc.convexHull(points.get(i), convexedPoints);
    points.set(i, convertIndexesToPoints(points.get(i), convexedPoints));
}

// sort points by size

MatOfPoint approxPoint = null;
for (MatOfPoint p : points) {
    MatOfPoint2f c = new MatOfPoint2f(p.toArray());
    double peri = Imgproc.arcLength(c, true);
    MatOfPoint2f approx = new MatOfPoint2f();
    Imgproc.approxPolyDP(c, approx, 0.1 * peri, true);

    if (approx.toArray().length == 4 && Imgproc.contourArea(p) >= (imageArea * 0.25)) {
        // this is the contour
    }
}

And the code to convert the detected contour to the full-sized image:

    double scaleWidth = imgMat.width() / previewSizeWidth;
double scaleHeight = imgMat.height() / previewSizeHeight;

MatOfPoint scaled = new MatOfPoint();
Core.multiply(detectedPoint, new Scalar(scaleWidth, scaleHeight), scaled);

// from scaled, sort points to get topLeft, topRight, ...

// calculate new width and height
double widthA = Math.sqrt(
        Math.pow(bottomRight.x - bottomLeft.x, 2) +
                Math.pow(bottomRight.y - bottomLeft.y, 2)
);
double widthB = Math.sqrt(
        Math.pow(topRight.x - topLeft.x, 2) +
                Math.pow(topRight.y - topLeft.y, 2)
);
double width = Math.max(widthA, widthB);

double heightA = Math.sqrt(
        Math.pow(topRight.x - bottomRight.x, 2) +
                Math.pow(topRight.y - bottomRight.y, 2)
);
double heightB = Math.sqrt(
        Math.pow(topLeft.x - bottomLeft.x, 2) +
                Math.pow(topLeft.y - bottomLeft.y, 2)
);
double height = Math.max(heightA, heightB);

// prepare mats for perspective and transform
MatOfPoint2f src = new MatOfPoint2f(
        topLeft,
        topRight,
        bottomRight,
        bottomLeft
);
MatOfPoint2f dest = new MatOfPoint2f(
        new Point(0, 0),
        new Point(width - 1, 0),
        new Point(width - 1, height - 1),
        new Point(0, height - 1)
);

// prepare mats for perspective and transform
MatOfPoint2f src = new MatOfPoint2f(
        topLeft,
        topRight,
        bottomRight,
        bottomLeft
);
MatOfPoint2f dest = new MatOfPoint2f(
        new Point(0, 0),
        new Point(width - 1, 0),
        new Point(width - 1, height - 1),
        new Point(0, height - 1)
);

// apply perspective and transform
Mat perspective = Imgproc.getPerspectiveTransform(src, dest);
Mat finalImage = new Mat((int) height, (int) width, CvType.CV_8UC1);
Imgproc.warpPerspective(image, finalImage, perspective, new Size(width, height));

Sorry for not attaching code earlier.

edit retag flag offensive close merge delete

Comments

Try taking low and high resolution pictures of a chessboard test pattern. Do they actually line up? It's possible the low resolution image has some cropping, not just downscaling.

Tetragramm gravatar imageTetragramm ( 2017-03-21 17:47:19 -0600 )edit

Thanks, will try that.

khrizt gravatar imagekhrizt ( 2017-03-22 03:02:56 -0600 )edit

The test with a chessboard gives rectangular shapes instead of square shapes and same margin problem. I've updated the question with those images to see if anyone can help me out. Thanks!

khrizt gravatar imagekhrizt ( 2017-03-22 03:15:40 -0600 )edit

Actually, without a single line of code it is quite guessing where your correction is going wrong. I have done exactly the same for page cropping and never met this issue once you indeed find the contour.

StevenPuttemans gravatar imageStevenPuttemans ( 2017-03-22 05:39:35 -0600 )edit

Sorry Steven, I've added some code to see if someone has some insight. Thanks!

khrizt gravatar imagekhrizt ( 2017-03-22 10:01:56 -0600 )edit

You say it has a preview, then it takes the high resolution image. Save both, before the warp perspective, and make sure they actually line up.

Tetragramm gravatar imageTetragramm ( 2017-03-22 21:09:21 -0600 )edit

Both images line up perfectly, and even the warp perspective on the preview gives a perfectly cropped and square image. I guess it can be something about the scale but I really don't know what. Thanks

khrizt gravatar imagekhrizt ( 2017-03-23 05:11:16 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-06-19 06:33:48 -0600

khrizt gravatar image

For those who find themselves in the same position, the problem was the division between integers results in an integer value instead of a double value. So this:

double scaleWidth = imgMat.width() / previewSizeWidth;
double scaleHeight = imgMat.height() / previewSizeHeight;

should be:

double scaleWidth = ((double) imgMat.width()) / ((double) previewSizeWidth);
double scaleHeight = ((double) imgMat.height()) / ((double) previewSizeHeight);

besides this the transportation works flawlessly.

edit flag offensive delete link more

Question Tools

1 follower

Stats

Asked: 2017-03-21 15:03:38 -0600

Seen: 367 times

Last updated: Jun 19 '17