How to classify image of object on a game board

asked 2020-05-01 21:53:58 -0600

Altheim gravatar image

updated 2020-05-04 22:01:39 -0600

UPDATE : I have tested implementation on PHash, ORB, AKAZE, Template Matching, Histogram Matching.

Imagine I have a squared game board the same as Match-3 game (namely Candy Crush, Puzzle and Dragon, etc), my objective is to detect the orb type on their screen and save the result as an array. I have a set of orbs image with transparent background as the source image.

Game board of P&D

Source orb images

I worked on capturing the board screen and splitting the image into chunks of small square image each containing the orb but with the board color in the background.

I tried Histogram matching (both 4 methods) but all return false result. Template Matching yield only some correct matches but mostly incorrect. Possibly due to the background color of the board in the captured image. And in some cases, if my source image is an orb in grayscale (most image is colored), I got this error while comparing my captured image (colored) with the grayscale source image.

OpenCV(3.4.3) Error: Assertion failed ((depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2) in void cv::matchTemplate(cv::InputArray, cv::InputArray, cv::OutputArray, int, cv::InputArray), file /build/3_4_pack-android/opencv/modules/imgproc/src/templmatch.cpp, line 1102
E/org.opencv.imgproc: imgproc::matchTemplate_11() caught cv::Exception: OpenCV(3.4.3) /build/3_4_pack-android/opencv/modules/imgproc/src/templmatch.cpp:1102: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'void cv::matchTemplate(cv::InputArray, cv::InputArray, cv::OutputArray, int, cv::InputArray)'

with this code:

baseImage = new Mat[69];
                baseImage[0] = Utils.loadResource(this, R.drawable.water, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //normal orb
                baseImage[1] = Utils.loadResource(this, R.drawable.w_s, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //shiny orb
                baseImage[2] = Utils.loadResource(this, R.drawable.w_w, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //weather orb
                baseImage[3] = Utils.loadResource(this, R.drawable.w_s_w, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //shiny weather orb
                baseImage[4] = Utils.loadResource(this, R.drawable.w_mono, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //monotone orb
                baseImage[5] = Utils.loadResource(this, R.drawable.w_s_mono, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //shiny monotone orb
                baseImage[6] = Utils.loadResource(this, R.drawable.w_target, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //targeted orb
                baseImage[7] = Utils.loadResource(this, R.drawable.w_i0, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //initial froze orb
                baseImage[8] = Utils.loadResource(this, R.drawable.w_i1, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //completely frozen orb
                baseImage[9] = Utils.loadResource(this, R.drawable.w_i2, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //second stage frozen orb
                baseImage[10] = Utils.loadResource(this, R.drawable.w_i3, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED); //last stage frozen orb

                baseImage[11] = Utils.loadResource(this, R.drawable.fire, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[12] = Utils.loadResource(this, R.drawable.f_s, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[13] = Utils.loadResource(this, R.drawable.f_w, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[14] = Utils.loadResource(this, R.drawable.f_s_w, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[15] = Utils.loadResource(this, R.drawable.f_mono, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[16] = Utils.loadResource(this, R.drawable.f_s_mono, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[17] = Utils.loadResource(this, R.drawable.f_target, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[18] = Utils.loadResource(this, R.drawable.f_i0, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[19] = Utils.loadResource(this, R.drawable.f_i1, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
                baseImage[20] = Utils.loadResource(this, R ...
(more)
edit retag flag offensive close merge delete