paper edge detection and perspective transform

asked 2019-05-24 02:47:04 -0500

crapthings gravatar image

updated 2019-05-24 03:13:02 -0500

before image

https://imgur.com/f190UFk

processed image

https://imgur.com/JkEhWkS

you can see the "processed image" has highlight, so the transform works bad.

any possible to make an rectangle that ignore that highlight area?

import os
import cv2
import numpy as np
from nanoid import generate

def processImage(imagepath, ext):
  img = cv2.imread(imagepath)

  hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
  h, s, v = cv2.split(hsv)

  # _, threshed = cv2.threshold(s, 50, 255, cv2.THRESH_BINARY_INV)
  _, threshed = cv2.threshold(s, 50, 255, cv2.THRESH_BINARY_INV)

  # cnts = cv2.findContours(threshed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
  cnts = cv2.findContours(threshed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
  canvas = img.copy()
  #cv2.drawContours(canvas, cnts, -1, (0, 255, 0), 1)

  cnts = sorted(cnts, key = cv2.contourArea)
  cnt = cnts[-1]
  print(cnt)

  arclen = cv2.arcLength(cnt, True)
  approx = cv2.approxPolyDP(cnt, 0.005 * arclen, True)
  cv2.drawContours(canvas, [cnt], -1, (255, 0, 0), 5, cv2.LINE_AA)
  cv2.drawContours(canvas, [approx], -1, (0, 0, 255), 1, cv2.LINE_AA)
  print(approx)
  approx = rectify(approx)
  pts2 = np.float32([[0, 0], [2480, 0], [2480, 3508], [0, 3508]])
  M = cv2.getPerspectiveTransform(approx, pts2)
  dst = cv2.warpPerspective(canvas, M, (2480, 3508))

  filename_output = generate() + ext
  cv2.imwrite('./static/' + filename_output, dst)

  topLeft, topRight, bottomRight, bottomLeft = approx
  topLeft = topLeft.tolist()
  topRight = topRight.tolist()
  bottomRight = bottomRight.tolist()
  bottomLeft = bottomLeft.tolist()

  return {
    'filename': './static/' + filename_output,
    'shape': img.shape,
    'approx': {
      'topLeft': topLeft,
      'topRight': topLeft,
      'bottomRight': bottomRight,
      'bottomLeft': bottomLeft,
    },
  }

def rectify(h):
  h = h.reshape((13, 2))
  hnew = np.zeros((4, 2), dtype = np.float32)
  add = h.sum(1)
  hnew[0] = h[np.argmin(add)]
  hnew[2] = h[np.argmax(add)]
  diff = np.diff(h, axis = 1)
  hnew[1] = h[np.argmin(diff)]
  hnew[3] = h[np.argmax(diff)]
  return hnew

add similar condition image

https://imgur.com/wDXtLsd

https://imgur.com/KAvOtdG

edit retag flag offensive close merge delete

Comments

1

can you please put your images HERE, not on bloody imgur (where it's not visible in several countries, due to shit content on that site) ? thank you..

berak gravatar imageberak ( 2019-05-24 06:00:29 -0500 )edit

Often machine vision challenges are not so much software as what I call staging. By staging I mean foreground, background, lighting, camera/subject position and orientation. In this case there is a glare that is saturating pixels and one page that is sticking out underneath to cause confusion. Put a single page on a black background with diffuse lighting and this problem gets much easier. The AIA basic certification has an entire section on lighting. https://www.visiononline.org/vision/c...

Chris gravatar imageChris ( 2019-05-25 06:58:41 -0500 )edit
1

Use adaptiveThreshold instead of the normal one. Once you find the right parameters of the adaptiveThreshold, your code should be more or less immune to highlights. Of course, it will not work in some extreme cases, but it should be ok for the case shown.

Witek gravatar imageWitek ( 2019-05-25 19:38:44 -0500 )edit

For this problem I would use a Hough line detector. Knowing that the paper is rectangular and the edges are mostly parallel and close to horizontal/vertical, you can filter the results to obtain the edge lines. The corners of the paper are the intersection of these lines.

kbarni gravatar imagekbarni ( 2019-05-27 04:52:45 -0500 )edit