Speeding up my code for object recognition
I am working on a multi-object recognition program. I have succeeded recognising two objects. However, the speed of my program is really slow and laggy. Can somebody please tell a way to speed the program up?
So far, I have seen that ORB is quite fast at matching features. However, my program is still too slow. I have also noticed that there are three for loops, possibly making the program slow down. Interestingly on YouTube, I have seen videos where the output is really smooth.
Is there a way to fix this? Thanks.
This is my code:
#include <stdio.h>
#include <iostream>
#include "opencv2/core/core.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/nonfree/features2d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/calib3d/calib3d.hpp"
using namespace std;
using namespace cv;
FlannBasedMatcher matcher(new flann::LshIndexParams(20, 10, 2));
void ttea(vector<KeyPoint> bwImgKP, Mat objectDS, Mat bwImgDS, vector<KeyPoint> objectKP, vector<Point2f> objectCRN, Mat colorImg, string text)
{
vector<vector<DMatch>> matches;
vector<DMatch> matchesGD;
vector<Point2f> obj;
vector<Point2f> scene;
vector<Point2f> sceneCRN(4);
Mat H;
if (bwImgKP.empty() || objectDS.empty() || bwImgDS.empty())
{
return;
}
matcher.knnMatch(objectDS, bwImgDS, matches, 2);
for (int i = 0; i < min(bwImgDS.rows - 1, (int)matches.size()); i++)
{
if ((matches[i][0].distance < 0.6*(matches[i][1].distance)) && ((int)matches[i].size() <= 2 && (int)matches[i].size() > 0))
{
matchesGD.push_back(matches[i][0]);
}
}
if (matchesGD.size() >= 4)
{
for (int i = 0; i < matchesGD.size(); i++)
{
obj.push_back(objectKP[matchesGD[i].queryIdx].pt);
scene.push_back(bwImgKP[matchesGD[i].trainIdx].pt);
}
H = findHomography(obj, scene, CV_RANSAC);
perspectiveTransform(objectCRN, sceneCRN, H);
line(colorImg, sceneCRN[0], sceneCRN[1], Scalar(255, 0, 0), 4);
line(colorImg, sceneCRN[1], sceneCRN[2], Scalar(255, 0, 0), 4);
line(colorImg, sceneCRN[2], sceneCRN[3], Scalar(255, 0, 0), 4);
line(colorImg, sceneCRN[3], sceneCRN[0], Scalar(255, 0, 0), 4);
putText(colorImg, text, sceneCRN[1], FONT_HERSHEY_DUPLEX, 1, Scalar(0, 0, 255), 1, 8);
}
}
int main()
{
OrbFeatureDetector detector;
OrbDescriptorExtractor extractor;
VideoCapture capture(0);
Mat object0 = imread("Much Ado About Nothing.jpg", CV_LOAD_IMAGE_GRAYSCALE);
vector<KeyPoint> object0KP;
detector.detect(object0, object0KP);
Mat object0DS;
extractor.compute(object0, object0KP, object0DS);
vector<Point2f> object0CRN(4);
object0CRN[0] = (cvPoint(0, 0));
object0CRN[1] = (cvPoint(object0.cols, 0));
object0CRN[2] = (cvPoint(object0.cols, object0.rows));
object0CRN[3] = (cvPoint(0, object0.rows));
Mat object1 = imread("Popular Science.jpg", CV_LOAD_IMAGE_GRAYSCALE);
vector<KeyPoint> object1KP;
detector.detect(object1, object1KP);
Mat object1DS;
extractor.compute(object1, object1KP, object1DS);
vector<Point2f> object1CRN(4);
object1CRN[0] = (cvPoint(0, 0));
object1CRN[1] = (cvPoint(object1.cols, 0));
object1CRN[2] = (cvPoint(object1.cols, object1.rows));
object1CRN[3] = (cvPoint(0, object1.rows));
while (true)
{
Mat bwImg;
Mat bwImgDS;
vector<KeyPoint> bwImgKP;
Mat colorImg;
capture.read(colorImg);
cvtColor(colorImg, bwImg, CV_BGR2GRAY);
detector.detect(bwImg, bwImgKP);
extractor.compute(bwImg, bwImgKP, bwImgDS);
ttea(bwImgKP, object0DS, bwImgDS, object0KP, object0CRN, colorImg, "Play");
ttea(bwImgKP, object1DS, bwImgDS, object1KP, object1CRN, colorImg, "Magazine");
imshow("Fish Smart", colorImg);
if (waitKey(1) == 27)
{
return 0;
}
}
}
feature matching is meant to find the pose of a known template object in another scene, you're the umpteenth person, who tries to abuse it for multiple object detection, but it will never work.
what do you mean?
However, on YouTube, I have seen people who has succeeded multiple object detection and their program is quite smooth...
anyone can cheat on youtube, you should forget about that site as input entirely.
if you want to build an object recognition system for multiple, arbitrary objects, you're on the wrong path here, you cannot use feature matching successfully for this, it does not know anything about "objects" in the 1st place. (all it has is corresponding points from 2 images) also, there is no proper way to compare the "detection" of one "object" to another.
maybe i misunderstand you entirely, but, did you want to build something like this ?