Not a god match

asked 2018-09-30 05:50:26 -0600

wiki gravatar image

Hello everyone!

It's my first time with opencCV and I'm trying to get the phone's camera to recognize a stored object whose image is stored in the assets folder.

When I find that objection, I want you to launch a new activty. My problem is that it launches the activity always, as soon as the camera appears it launches the new activity, so the match it makes is not very good.

I leave the code, thanks:

public class Main3Activity extends Activity implements CameraBridgeViewBase.CvCameraViewListener2 {

private static final String TAG = "OCVSample::Activity";
private int w, h;
private CameraBridgeViewBase mOpenCvCameraView;
TextView tvName;
Scalar RED = new Scalar(255, 0, 0);
Scalar GREEN = new Scalar(0, 255, 0);
FeatureDetector detector;
DescriptorExtractor descriptor;
DescriptorMatcher matcher;
Mat descriptors2,descriptors1;
Mat img1;
MatOfKeyPoint keypoints1,keypoints2;

static {
    if (!OpenCVLoader.initDebug())
        Log.d("ERROR", "Unable to load OpenCV");
    else
        Log.d("SUCCESS", "OpenCV loaded");
}

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
    @Override
    public void onManagerConnected(int status) {
        switch (status) {
            case LoaderCallbackInterface.SUCCESS: {
                Log.i(TAG, "OpenCV loaded successfully");
                mOpenCvCameraView.enableView();
                try {
                    initializeOpenCVDependencies();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            break;
            default: {
                super.onManagerConnected(status);
            }
            break;
        }
    }
};

private void initializeOpenCVDependencies() throws IOException {
    mOpenCvCameraView.enableView();
    detector = FeatureDetector.create(FeatureDetector.ORB);
    descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);
    matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
    img1 = new Mat();
    AssetManager assetManager = getAssets();
    InputStream istr = assetManager.open("ajustes.png");
    Bitmap bitmap = BitmapFactory.decodeStream(istr);
    Utils.bitmapToMat(bitmap, img1);
    Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGB2GRAY);
    img1.convertTo(img1, 0); //converting the image to match with the type of the cameras image
    descriptors1 = new Mat();
    keypoints1 = new MatOfKeyPoint();
    detector.detect(img1, keypoints1);
    descriptor.compute(img1, keypoints1, descriptors1);

}


public Main3Activity() {

    Log.i(TAG, "Instantiated new " + this.getClass());
}

/**
 * Called when the activity is first created.
 */
@Override
public void onCreate(Bundle savedInstanceState) {

    Log.i(TAG, "called onCreate");
    super.onCreate(savedInstanceState);
    getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
    setContentView(R.layout.activity_main3);
    mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_java_surface_view);
    mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
    mOpenCvCameraView.setCvCameraViewListener(this);
    tvName = (TextView) findViewById(R.id.text1);

}

@Override
public void onPause() {
    super.onPause();
    if (mOpenCvCameraView != null)
        mOpenCvCameraView.disableView();
}

@Override
public void onResume() {
    super.onResume();
    if (!OpenCVLoader.initDebug()) {
        Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
        OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);
    } else {
        Log.d(TAG, "OpenCV library found inside package. Using it!");
        mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
    }
}

public void onDestroy() {
    super.onDestroy();
    if (mOpenCvCameraView != null)
        mOpenCvCameraView.disableView();
}

public void onCameraViewStarted(int width, int height) {
    w = width;
    h = height;
}

public void onCameraViewStopped() {
}

public Mat recognize(Mat aInputFrame) {

    Imgproc.cvtColor(aInputFrame, aInputFrame, Imgproc.COLOR_RGB2GRAY);
    descriptors2 = new Mat();
    keypoints2 = new MatOfKeyPoint();
    detector.detect(aInputFrame, keypoints2);
    descriptor.compute(aInputFrame, keypoints2, descriptors2);

    // Matching
    MatOfDMatch matches = new MatOfDMatch();
    if (img1.type() == aInputFrame.type()) {
        matcher.match(descriptors1, descriptors2, matches);

    } else {
        return aInputFrame;
    }
    List<DMatch> matchesList = matches.toList();

    Double max_dist = 0.0;
    Double min_dist = 100.0;

    for (int i = 0; i < matchesList.size(); i++) {
        Double dist = (double) matchesList.get(i).distance;
        if (dist < min_dist)
            min_dist = dist;
        if (dist > max_dist)
            max_dist = dist;
    }

    LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
    for (int i = 0; i < matchesList.size(); i++) {
        if ...
(more)
edit retag flag offensive close merge delete

Comments

to recognize a stored object

sorry to say so, but it somewhat looks you were mislead thinking, this would be some "object detection"

(it isn't, you may use it to find a homography for a known part of a scene, but your code is not doing that)

please have a look at the docs again

berak gravatar imageberak ( 2018-09-30 05:58:24 -0600 )edit

@berak do not worry, thanks for the information, I will try to use the documents you leave me, at first glance compare two images, I imagine that one of them can be obtained directly from the camera. The imporante of the documents that you leave me is to use SURF features no

In another order of things, do you have documentation so that you can compare with an image compare it with many? For example, have 10 images in the assets folder and let the caname recognize which one you are seeing

wiki gravatar imagewiki ( 2018-09-30 06:09:01 -0600 )edit

then, you basically start a new Intent every time you see a valid input image (which is outright silly for sure not what you wanted)

berak gravatar imageberak ( 2018-09-30 06:10:48 -0600 )edit

we DO have proper object detection using cnn's but that is an entirely different story

berak gravatar imageberak ( 2018-09-30 06:12:42 -0600 )edit
1

And how do I enter the images that you have to look for? for example, if they were very small, 1000 images, can be done with an array or similar?

Sorry but it's my first time with openCV and I also have little with android and I'm a bit clumsy. Do you know any place where you can see some code?

wiki gravatar imagewiki ( 2018-09-30 06:15:21 -0600 )edit

no, cnns work different. you train those on some known objects, then later you predict which of those it is in your test image.

you also can use 2d features to match images, but that needs a different matching process, and might not be easily possible on android, see related question here and here

berak gravatar imageberak ( 2018-09-30 06:23:33 -0600 )edit

So, for what I want it to be, with the camera you can recognize the screen, the mouse, the keyboard, the lamp, the table, the window, objects to which I have photographed them, what should I use?

Is it possible with Homography to pass these 6 objects and that the algorithm compares the image that it takes from the camera with the photo it has of the object?

It's very similar to that, page 5 of 12 of the pdf (http://44jaiio.sadio.org.ar/sites/default/files/sts252-263.pdf)

If homography is not worth ... what can I use? tensorflow?

Thank you very much for answering everything and so fast

wiki gravatar imagewiki ( 2018-09-30 06:52:47 -0600 )edit

pdf in spanish (useless) but at least you're trying to read up ;)

the homography will give you the pose/position of a known thing, that's not what you wanted (which is more classification, imho) it can't tell you what it was, or if it was NOT there.

i think,y ou should first try with the mobilenet sample, and then, maybe try to train your own network using tensorflow or such (you can even use google's colab for this for free !)

berak gravatar imageberak ( 2018-09-30 07:31:50 -0600 )edit

but, since it's sunday afternoon, what about trying the most simple approach at comparing images ?

  • load your image as grayscale, resize() to the gallery image size
  • for each image in the gallery do
    • absdiff(img,gallery,diff);
    • double distance = sum(diff)[0];
  • winner is the one with the smallest distance ;)


easy to do, it's just a few lines. it won't win any competition. but you've done the minimal thing already, and you'll see a lot from that silly little thing.

berak gravatar imageberak ( 2018-09-30 07:48:43 -0600 )edit

Thank you very much, the help of using tensorflow has helped me a lot, I found this tutorial, (in English XD) that can be the beginning of what I intend (I leave the link in case it helps someone)

wiki gravatar imagewiki ( 2018-09-30 13:56:30 -0600 )edit