Ask Your Question

object recognition in real time on Android with FAST detector

asked 2013-03-13 03:46:21 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >50 is welcome to improve it.

The aim of this work is to extract real-time key points from an existing image in a video scene processFrame has the execution that does not work, due to match, is that I can just display the key points of correspondence in the form of circles on the image in real time

   class Sample1View extends SampleViewBase {
    public static final int     VIEW_MODE_RGBA   = 0;
    public static final int  VIEW_MODE_BLUE  = 1; 
    public static final int  VIEW_MODE_YELLOW = 2;
    public static final int  VIEW_MODE_DE = 3;
    private Mat mYuv;
    private Mat mRgba;
    private Mat mGraySubmat;
    private Mat mResult;
    private Mat mIntermediateMat;
 private Bitmap mBitmap;
 private int mViewMode;
 private Mat mColor;
 private Mat mHsv;
 TimingLogger timings;
   private Mat img1;
    private Mat descriptors;
    private MatOfKeyPoint keypoints;
    private FeatureDetector detector;
    private DescriptorExtractor descriptor;
    private DescriptorMatcher matcher;

 private static final String TAG ="Sample::View";

    public Sample1View(Context context) {
        mViewMode = VIEW_MODE_RGBA;

 try {
 img1=Utils.loadResource(getContext(), R.drawable.wings);
 } catch (IOException e) {
 // TODO Auto-generated catch block
 Log.w("Activity::LoadResource","Unable to load resource R.drawable.wings");
        descriptors = new Mat();
        keypoints = new MatOfKeyPoint();
        detector = FeatureDetector.create(FeatureDetector.FAST);
        detector.detect(img1, keypoints);
        descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);
        descriptor.compute(img1, keypoints, descriptors);
        matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

 protected void onPreviewStarted(int previewWidth, int previewHeight) {
     Log.i(TAG, "preview Started");
     synchronized (this) {
         mYuv = new Mat(getFrameHeight() + getFrameHeight() / 2, getFrameWidth(), CvType.CV_8UC1);
         mGraySubmat = mYuv.submat(0, getFrameHeight(), 0, getFrameWidth());
         mRgba = new Mat();
         mIntermediateMat = new Mat();
         mBitmap = Bitmap.createBitmap(previewWidth, previewHeight, Bitmap.Config.ARGB_8888); 
         mHsv = new Mat();
         mColor = new Mat();
         mResult = new Mat();

 protected void onPreviewStopped() {
     Log.i(TAG, "preview Stopped");
     if(mBitmap != null) {

  synchronized (this) {
            // Explicitly deallocate Mats
            if (mYuv != null)
            if (mRgba != null)
            if (mGraySubmat != null)
            if (mIntermediateMat != null)
            mYuv = null;
            mRgba = null;
            mGraySubmat = null;
            mIntermediateMat = null;
            if (mResult != null)
            if (mHsv != null)
            if (mColor != null)
            mColor = null;
         mResult = null;
         mHsv = null;
    protected Bitmap processFrame(byte[] data) {

        mYuv.put(0, 0, data);
        final int viewMode = mViewMode;


      MatOfKeyPoint mKeyPoints = new MatOfKeyPoint();
    MatOfDMatch  matches = new MatOfDMatch();
      detector.detect(mGraySubmat, mKeyPoints);
      descriptor.compute(mGraySubmat, mKeyPoints, mIntermediateMat);
      matcher.match(mIntermediateMat, descriptors, matches);
      mIntermediateMat2.create(resultSize, CvType.CV_8UC1);

      Features2d.drawMatches(mGraySubmat, mKeyPoints, mGraySubmat, mKeyPoints, matches, mIntermediateMat2);
      Imgproc.resize(mIntermediateMat2, mIntermediateMat2, mRgba.size());
      Imgproc.cvtColor(mIntermediateMat2, mRgba, Imgproc.COLOR_RGBA2BGRA, 4);
    Bitmap bmp = mBitmap;

    try {
      Utils.matToBitmap(mRgba, bmp);

    } catch(Exception e) {
        Log.e("org.opencv.samples.*", "Utils.matToBitmap() throws an exception: " + e.getMessage());
        bmp = null;
    return bmp;

    public void setViewMode(int viewMode) {
     mViewMode = viewMode;
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2013-03-14 03:44:43 -0600

It is still not clear to me what is exactly your problem. I guess the above code doesn't work. Can you try first to follow the following tutorials on keypoint matching, to see if you get it to work?

This code is actually correct and these samples do work when followed correctly. Show us some errors if it doesnt:

edit flag offensive delete link more


I am getting the following error while using OpenCV

OpenCV Error: Assertion failed (i1 >= 0 && i1 < static_cast<int>(keypoints1.size())) in void cv::drawMatches(const cv::Mat&, const std::vector<cv::KeyPoint>&, const cv::Mat&, const std::vector<cv::KeyPoint>&, const std::vector<cv::DMatch>&, cv::Mat&, const Scalar&, const Scalar&, const std::vector<char>&, int), file /home/reports/ci/slave/50-SDK/opencv/modules/features2d/src/draw.cpp, line 207

Marwen Z gravatar imageMarwen Z ( 2013-03-14 04:04:45 -0600 )edit

the process is simple i try to draw matching between an existing image and frame from camera

Marwen Z gravatar imageMarwen Z ( 2013-03-14 04:07:19 -0600 )edit

Reading the error, it states that you are feeding the wrong type of keypoint sets. Go to the feature matching algorithm above and try to follow it! It does work, as I have done it multiple times.

StevenPuttemans gravatar imageStevenPuttemans ( 2013-03-14 04:11:54 -0600 )edit

but those methods detector.detect(); descriptor.compute(); and Features2d.drawMatches ACCEPT only MatOfKeyPoint type and not List<KeyPoint> ok i'm going to try all solution thnx

Marwen Z gravatar imageMarwen Z ( 2013-03-14 04:21:09 -0600 )edit

@StevenPuttemans i have another solution and i hope that you help me when I get matches of the two images is that I can just draw commonalities in just the image of the scene other saying at each correspondence of the two images I draw circles in the scene PS :matcher.match(mIntermediateMat, descriptors, matches); its working on emulator but not on device

Marwen Z gravatar imageMarwen Z ( 2013-03-14 06:22:55 -0600 )edit

I have zero experience with running openCV on mobile devices. I do know there are a lot of device specific settings for each manufacturer, even if they all use the same version of for example android. I suggest rearranging your new problem in a clear new question and waiting for a possible response!

StevenPuttemans gravatar imageStevenPuttemans ( 2013-03-14 06:47:51 -0600 )edit

@StevenPuttemans after the recognition phase I want to insert a 3D object on the object detected, so what method to do this ? i don't wont to use OpenGL ES

Marwen Z gravatar imageMarwen Z ( 2013-04-29 08:58:25 -0600 )edit

Make a new topic for your question, this one is handled. Also accept this one if you finished the detection/recognition part.

StevenPuttemans gravatar imageStevenPuttemans ( 2013-04-29 09:29:26 -0600 )edit

Question Tools

1 follower


Asked: 2013-03-13 03:46:21 -0600

Seen: 8,884 times

Last updated: Mar 14 '13