How to do a perspective transformation of an image which is missing corners using opencv java

asked 2020-06-29 07:01:47 -0600

updated 2020-10-10 20:29:09 -0600

6772 ●3 ●48 ●79 https://github.com/stu...

I am trying to build a document scanner using openCV. I am trying to auto crop an uploaded image. I have few use cases where there is a gap in the border when the document is out of frame(captured image).

Ex image

$C:\fakepath\unnamed.jpg$

Below is the canny edge detection of the given image.

$C:\fakepath\canned.jpg$

The borders are missing here and findContours does not return me proper results due to this. How can I handle such images.

Both automatic canny edge detection as well as dilate does not work in such cases because it can join only small edges.

Also few documents might have only 2 sides or 3 sides captured using camera and how can we crop the other areas which is not required.

Example Image:

$C:\fakepath\unamed2.jpg$

Is there any specific technique for handling such documents? Please suggest few ideas.

Source Code: gist.github.com/sureshbabuinfo/f398a7...

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2020-06-29 17:14:04 -0600

holger
364 ●4 ●13

How to crop in java:
https://answers.opencv.org/question/1...

Most difficult part would be to get the bounding box of the whole text reliably. In theory you could just take the most left, top, right, bottom inner countour, and voila you have a rect. But i am afraid find contours could also detect some noise(things which are no text). Look at your image on the very top right - the little hole in the paper. But maybe thats no problem for your use case - just try out.

edit flag offensive delete link

Comments

Im new to these concepts. Can you elaborate little bit?

sureshbabu ( 2020-06-30 06:02:36 -0600 )edit

I am not sure - its still a bit unclear what you want to do. In general cropping means you have a region of interest you want to extract - the short term is ROI. You then extract that region from the image - this is called cropping.

I dont know what your ROI is - The whole text (lets call it document here) with some padding? Please clarify what you want to do - you asked for cropping and you got an answer for this(check the link - its working for java)

I assumed that you want to detect the document. You already got the contours for words. So my suggestion was to take the most left, top , bottom, right word and form an rectangle out of it (base math- right?). And thats your roi for cropping.

holger ( 2020-06-30 08:14:12 -0600 )edit

My use case is, user will take a picture of his document and upload it. It can be either ID card or an A4 sheet etc. When user captures the image properly where the four edges of the document he captures are properly seen in the image it is well and good. But when he captures an image like above, im not able to find four points of the edges to do a perspective transformation.

sureshbabu ( 2020-06-30 08:43:45 -0600 )edit

would measuring the angle on the line of the outer contour work? Now that i understand what you want to do (perspective trandsform - most likely into a birds view), i must admit i am not really qualified for this - maybe raise a new question and this time give a better description what you want to do - what you already have done(some code snippets), hopefully u get some better answers.

holger ( 2020-06-30 11:06:51 -0600 )edit

@sureshbabu. I understood. I am not familiar with java. Usually, I do with python. But it will convert to java too. Your question is cleared to me able to find four points of the edges to do a perspective transformation.. Yes it will do.

supra56 ( 2020-07-01 22:05:12 -0600 )edit

@supra56 Im not able to find four points for the given images above. My question is how can we do that. Can you post it in python so that I can convert it to Java

sureshbabu ( 2020-07-02 07:44:09 -0600 )edit

Thank you @supra56 . I will try that and get back.

sureshbabu ( 2020-07-02 09:26:51 -0600 )edit

@supra56 Are you doing it based on user selection of four points? Because I want to do it automatically without any user interaction. I want to get the four points somehow.

sureshbabu ( 2020-07-02 09:32:38 -0600 )edit

Yes. Using mouse event to selected 4 points and press enter

supra56 ( 2020-07-02 09:39:41 -0600 )edit

Another one is draw rectangle and crop it. The 4-points and rectangle is best solution widely used and nothing more than.

supra56 ( 2020-07-02 09:40:58 -0600 )edit

see more comments

answered 2020-07-02 08:29:00 -0600

supra56

943 ●9 ●6

@sureshbabu. Here is code. So that you can convert to java:

#!/usr/bin/python37
#OpenCV 4.3.0, Raspberry pi3B/+, 4b/4g/8g, Thonny 3.7.3
#Date: 2nd July, 2020

import cv2
import numpy as np
from utils import get_four_points

if __name__ == '__main__' :

    # Read in the image.
    im_src = cv2.imread("paper.jpg")
    im_src =cv2.resize(im_src, (640, 480))

    # Destination image
    size = (640,480,3)
    im_dst = np.zeros(size, np.uint8)   
    pts_dst = np.array(
                       [
                        [0,0],
                        [size[0] - 1, 0],
                        [size[0] - 1, size[1] -1],
                        [0, size[1] - 1 ]
                        ], dtype=float
                       )

    print( '''
        Click on the four corners of the book -- top left first and
        bottom left last -- and then hit ENTER''')   
    # Show image and wait for 4 clicks.
    cv2.imshow("Image", im_src)
    pts_src = get_four_points(im_src);

    # Calculate the homography
    h, status = cv2.findHomography(pts_src, pts_dst)

    # Warp source image to destination
    im_dst = cv2.warpPerspective(im_src, h, size[0:2])

    # Show output
    cv2.imshow("Image", im_dst)
    cv2.waitKey(0)

Output:

image description

U have to rotate on editor. Becuase of paper resolution 3680x2160.

edit flag offensive delete link

add a comment

How to do a perspective transformation of an image which is missing corners using opencv java

2 answers

Comments

Links

Question Tools

Stats

Related questions

How to do a perspective transformation of an image which is missing corners using opencv java edit

2 answers

Comments

Links

Question Tools

Stats

Related questions

How to do a perspective transformation of an image which is missing corners using opencv java