Sorry about the gigantic post :P
First Question: The ViewLoop()
member function does not output anything to the namedWindow()
(simply a grey window with a loading cursor) despite the program still doing the facial rec...What is causing this?
Second Question: I've noticed this is a pretty common thing around here with non-realtime playback from a VideoCapture
being routed to a namedWindow
, so apologies if this appears repetitive.
I followed the tutorial for the FaceRecognizer module and despite the code being pretty terrible, the playback is far from real-time (more like 10-15 fps). From research, I've added the WINDOW_OPENGL
flag to the namedWindow()
as apparently it improves performance. I also came across an idea that I could split my program into 2 threads: the first takes the camera input and does the face recognition on the cv::Mat
, and the second handles the imshow()
function (which apparently is computationally heavy). Something about mutex
's came up as well. How would I go about this?
Would this give enough of a performance boost to make the recognition real-time? If not, what else could I do?
Here is my code (with the bug I described earlier) that I have prepared to be threaded (I'm not too familiar with c++ multithreading primarily since NO ONE can explain it properly to me -_-) and any other improvements are welcome!
FaceDetect.h
#ifndef FACEDETECT_H
#define FACEDETECT_H
#include "opencv2/core.hpp"
#include "opencv2/face.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/objdetect.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
#include <thread>
class FaceDetect {
public:
FaceDetect();
~FaceDetect();
static void ReadCSV(const std::string& filename, std::vector<cv::Mat>& images, std::vector<int>& labels, char separator = ';');
void init();
// The fast loop?
void FaceDetectLoop();
// The slow loop?
void ViewLoop();
private:
cv::VideoCapture m_cap;
int m_deviceID = 0;
int m_imgWidth, m_imgHeight;
std::string m_haarPath = "data/haarcascade_frontalface_default.xml";
std::string m_csvPath = "data/faceCSV.txt";
cv::Mat m_currFrame;
cv::Mat m_original;
std::vector<cv::Mat> m_images;
std::vector<int> m_labels;
cv::Ptr<cv::face::FaceRecognizer> m_model = cv::face::createFisherFaceRecognizer();
cv::CascadeClassifier m_haarCascade;
};
#endif // !FACEDETECT_H
FaceDetect.cpp
#include "FaceDetect.h"
using namespace std;
using namespace cv;
using namespace cv::face;
FaceDetect::FaceDetect() {
}
FaceDetect::~FaceDetect() {
}
//Read in data from CSV file (can be .txt, .csv, etc.)
void FaceDetect::ReadCSV(const std::string & filename, std::vector<cv::Mat>& images, std::vector<int>& labels, char separator) {
std::ifstream file(filename.c_str(), ifstream::in);
if (!file) {
string error_message = "No valid input file was given, please check the given filename.";
CV_Error(CV_StsBadArg, error_message);
}
string line, path, classlabel;
while (getline(file, line)) {
stringstream liness(line);
getline(liness, path, separator);
getline(liness, classlabel);
if (!path.empty() && !classlabel.empty()) {
images.push_back(imread(path, 0));
labels.push_back(atoi(classlabel.c_str()));
}
}
return;
}
void FaceDetect::init() {
m_cap.open(0);
if (!m_cap.isOpened()) {
cerr << "Capture Device ID " << m_deviceID << "cannot be opened." << endl;
return;
}
// Read in the data
try {
ReadCSV(m_csvPath, m_images, m_labels);
}
catch (cv::Exception& e) {
cerr << "Error opening file \"" << m_csvPath << "\". Reason: " << e.msg << endl;
exit(1);
}
//train recognizer ...
(more)