Ask Your Question
-1

Speeding up my code for object recognition

asked 2017-06-19 02:04:30 -0500

kimchiboy03 gravatar image

I am working on a multi-object recognition program. I have succeeded recognising two objects. However, the speed of my program is really slow and laggy. Can somebody please tell a way to speed the program up?

So far, I have seen that ORB is quite fast at matching features. However, my program is still too slow. I have also noticed that there are three for loops, possibly making the program slow down. Interestingly on YouTube, I have seen videos where the output is really smooth.

Is there a way to fix this? Thanks.

This is my code:

#include <stdio.h>
#include <iostream>
#include "opencv2/core/core.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/nonfree/features2d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/calib3d/calib3d.hpp"

using namespace std;
using namespace cv;

FlannBasedMatcher matcher(new flann::LshIndexParams(20, 10, 2));

void ttea(vector<KeyPoint> bwImgKP, Mat objectDS, Mat bwImgDS, vector<KeyPoint> objectKP, vector<Point2f> objectCRN, Mat colorImg, string text)
{
    vector<vector<DMatch>> matches;
    vector<DMatch> matchesGD;
    vector<Point2f> obj;
    vector<Point2f> scene;
    vector<Point2f> sceneCRN(4);
    Mat H;

    if (bwImgKP.empty() || objectDS.empty() || bwImgDS.empty())
    {
        return;
    }

    matcher.knnMatch(objectDS, bwImgDS, matches, 2);

    for (int i = 0; i < min(bwImgDS.rows - 1, (int)matches.size()); i++)
    {
        if ((matches[i][0].distance < 0.6*(matches[i][1].distance)) && ((int)matches[i].size() <= 2 && (int)matches[i].size() > 0))
        {
            matchesGD.push_back(matches[i][0]);
        }
    }

    if (matchesGD.size() >= 4)
    {
        for (int i = 0; i < matchesGD.size(); i++)
        {
            obj.push_back(objectKP[matchesGD[i].queryIdx].pt);
            scene.push_back(bwImgKP[matchesGD[i].trainIdx].pt);
        }

        H = findHomography(obj, scene, CV_RANSAC);

        perspectiveTransform(objectCRN, sceneCRN, H);

        line(colorImg, sceneCRN[0], sceneCRN[1], Scalar(255, 0, 0), 4);
        line(colorImg, sceneCRN[1], sceneCRN[2], Scalar(255, 0, 0), 4);
        line(colorImg, sceneCRN[2], sceneCRN[3], Scalar(255, 0, 0), 4);
        line(colorImg, sceneCRN[3], sceneCRN[0], Scalar(255, 0, 0), 4);
        putText(colorImg, text, sceneCRN[1], FONT_HERSHEY_DUPLEX, 1, Scalar(0, 0, 255), 1, 8);
    }
}

int main()
{
    OrbFeatureDetector detector;
    OrbDescriptorExtractor extractor;

    VideoCapture capture(0);

    Mat object0 = imread("Much Ado About Nothing.jpg", CV_LOAD_IMAGE_GRAYSCALE);

    vector<KeyPoint> object0KP;
    detector.detect(object0, object0KP);

    Mat object0DS;
    extractor.compute(object0, object0KP, object0DS);

    vector<Point2f> object0CRN(4);
    object0CRN[0] = (cvPoint(0, 0));
    object0CRN[1] = (cvPoint(object0.cols, 0));
    object0CRN[2] = (cvPoint(object0.cols, object0.rows));
    object0CRN[3] = (cvPoint(0, object0.rows));

    Mat object1 = imread("Popular Science.jpg", CV_LOAD_IMAGE_GRAYSCALE);

    vector<KeyPoint> object1KP;
    detector.detect(object1, object1KP);

    Mat object1DS;
    extractor.compute(object1, object1KP, object1DS);

    vector<Point2f> object1CRN(4);
    object1CRN[0] = (cvPoint(0, 0));
    object1CRN[1] = (cvPoint(object1.cols, 0));
    object1CRN[2] = (cvPoint(object1.cols, object1.rows));
    object1CRN[3] = (cvPoint(0, object1.rows));

    while (true)
    {
        Mat bwImg;
        Mat bwImgDS;
        vector<KeyPoint> bwImgKP;

        Mat colorImg;
        capture.read(colorImg);

        cvtColor(colorImg, bwImg, CV_BGR2GRAY);

        detector.detect(bwImg, bwImgKP);
        extractor.compute(bwImg, bwImgKP, bwImgDS);

        ttea(bwImgKP, object0DS, bwImgDS, object0KP, object0CRN, colorImg, "Play");
        ttea(bwImgKP, object1DS, bwImgDS, object1KP, object1CRN, colorImg, "Magazine");

        imshow("Fish Smart", colorImg);

        if (waitKey(1) == 27)
        {
            return 0;
        }
    }
}
edit retag flag offensive close merge delete

Comments

feature matching is meant to find the pose of a known template object in another scene, you're the umpteenth person, who tries to abuse it for multiple object detection, but it will never work.

berak gravatar imageberak ( 2017-06-19 02:39:19 -0500 )edit

what do you mean?

kimchiboy03 gravatar imagekimchiboy03 ( 2017-06-19 02:42:46 -0500 )edit

However, on YouTube, I have seen people who has succeeded multiple object detection and their program is quite smooth...

kimchiboy03 gravatar imagekimchiboy03 ( 2017-06-19 02:43:48 -0500 )edit

anyone can cheat on youtube, you should forget about that site as input entirely.

berak gravatar imageberak ( 2017-06-19 02:46:23 -0500 )edit

if you want to build an object recognition system for multiple, arbitrary objects, you're on the wrong path here, you cannot use feature matching successfully for this, it does not know anything about "objects" in the 1st place. (all it has is corresponding points from 2 images) also, there is no proper way to compare the "detection" of one "object" to another.

maybe i misunderstand you entirely, but, did you want to build something like this ?

berak gravatar imageberak ( 2017-06-19 02:50:16 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2017-06-19 03:22:11 -0500

berak gravatar image

though i think, you're on the wrong path with the purpose of it, there are ways to speed up feature matching

// 1. profile, don't guess !
int64 t0 = getTickCount();
some_operation(input, output);
int64 t1 = getTickCount();
double seconds = (t1-t0)/getTickFrequency();
cerr << "some_operation took: " << seconds << " seconds." << endl;

// 2. use detectAndCompute(), not single detect() and compute() calls. 
// (it has to compute the features for the detection anyway, avoid wouble work)
OrbFeatureDetector detector; // no seperate extractor needed.
detector.detectAndCompute(image, Mat(), keypoints, descriptors);

// 3. restrict number of keypoints (if you can allow it, quality.)
// http://docs.opencv.org/2.4.13/modules/features2d/doc/feature_detection_and_description.html#orb-orb
// please also see the remarks on other optimizations there, scaleFactor, scoreType
OrbFeatureDetector detector(500); // 500 enough ? experiment.

// 4. work on smaller images:
pyrDown(image, image); // half size, 1/4 area to check.
detector.detectAndCompute(image, Mat(), keypoints, descriptors);
edit flag offensive delete link more

Comments

Haha thank you for your post... However, I have decided to use the haar classifier method instead (and making my own .xml files) for object detection since you say its abusing the system. Thank you though :)

kimchiboy03 gravatar imagekimchiboy03 ( 2017-06-20 00:00:57 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-06-19 02:04:30 -0500

Seen: 125 times

Last updated: Jun 19 '17