Revision history [back]

I'm new to OpenCV and computer vision, so maybe you can push me in the right way.

The task is: to detect the type of document (from a certain number of known documents) in a real-time with the help of camera in the android device. Yet without OCR, just type recognition.

Examples of documents:
- Example 1
- Example 2
- Example 3

Documents always have a strict pattern of data placement.

What I've already tried and what are the main ideas.

1. As you see, all documents have a logo of a company. According to the logo and it's position we can filter a significant amount of patterns to search.

2. At first I've tried Template Matching.
   Pros: none.
   Cons: very slow and has bad results in my case. The bad thing it's moving a template picture pixel by pixel along the picture (from a camera) so it can't do anything if the logo in template and the logo in a picture (from a camera) have different resolutions. So it seems it's not suitable for my needs.

3. Then I've tried Feature Matching with FLANN (but on PC).
   Pros: really good match results.
   Match 1
   Match 2
   Cons: very slow. It takes a long time to detect, compute keypoints and perform the match. It seems it's not suitable for real-time needs or I'm doing something wrong.

4. So I need really simple and effective idea to detect types of documents fast. And adding new types of documents shouldn't decrease performance significantly.

   Now the idea is...

5. To get all contours first.
   Finding contours in your image. Maybe process image with Canny Edge Detector previously, it's doing a great job – one, two. The bad is it decreases finally performance significantly. On Nexus 5, on 1280x960 sample, Canny is working for 250-350ms, findContours (after Canny) – up to 60ms. So the result is 3 fps, which is not very responsive. Maybe to try some threshold functions here to filter unnecessary data.

6. Then try to convert contours to rectangles.

7. And according to rectangles sizes and positions try to recognize the document type.
   So try to match such document to something looking like this pattern (roughly).

   The bad thing, I think it will be hard and not very accurate to make rectangles from contours sometimes. Especially to make rectangles from blocks of text, diagrams or barcodes.

Any advices, ideas which way should I move?

I'm new to OpenCV and computer vision, so maybe you can push me in the right way.

The task is: to detect the type of document (from a certain number of known documents) in a real-time with the help of camera in the android device. Yet without OCR, just type recognition.

Examples of documents:
- Example 1
- Example 2
- Example 3

Documents always have a strict pattern of data placement.

What I've already tried and what are the main ideas.

1. As you see, all documents have a logo of a company. According to the logo and it's position we can filter a significant amount of patterns to search.

• At first I've tried Template Matching.
  Pros: none.
  Cons: very slow and has bad results in my case. The bad thing it's moving a template picture pixel by pixel along the picture (from a camera) so it can't do anything if the logo in template and the logo in a picture (from a camera) have different resolutions. So it seems it's not suitable for my needs.

• Then I've tried Feature Matching with FLANN (but on PC).
  Pros: really good match results.
  Match 1
  Match 2
  Cons: very slow. It takes a long time to detect, compute keypoints and perform the match. It seems it's not suitable for real-time needs or I'm doing something wrong.

2. So I need really simple and effective idea to detect types of documents fast. And adding new types of documents shouldn't decrease performance significantly.

   Now the idea is...

• To get all contours first.
  Finding contours in your image. Maybe process image with Canny Edge Detector previously, it's doing a great job – one, two. The bad is it decreases finally performance significantly. On Nexus 5, on 1280x960 sample, Canny is working for 250-350ms, findContours (after Canny) – up to 60ms. So the result is 3 fps, which is not very responsive. Maybe to try some threshold functions here to filter unnecessary data.

• Then try to convert contours to rectangles.

• And according to rectangles sizes and positions try to recognize the document type.
  So try to match such document to something looking like this pattern (roughly).

The bad thing, I think it will be hard and not very accurate to make rectangles from contours sometimes. Especially to make rectangles from blocks of text, diagrams or barcodes.

Any advices, ideas which way should I move?