I try to create an interactive projector using the method of laser pointer detecting by color, form and saturation. The projection itself is not static, it is dynamic and has a lot of white elements. It is also very important to exclude the possibility of a wrong response to the metallic light glares. Here is the code i tried to use to detect the laser with Visual C++:
#include <opencv2\core\core.hpp>
#include <opencv2\highgui\highgui.hpp>
#include <opencv2\imgproc\imgproc.hpp>
#include <iostream>
#include <Windows.h>
int main() {
cv::VideoCapture capWebcam(0);
// get screen resolution, which will be used in future
int widthx = GetSystemMetrics(SM_CXSCREEN);
int heighty = GetSystemMetrics(SM_CYSCREEN);
std::cout << (widthx) << " X " << (heighty) << "\n\n";
// webcam access
if (capWebcam.isOpened() == false) {
std::cout << "Error: Webcam don't work or can't be accessed\n\n";
return(1);
}
// creating various necessary staff
cv::Mat matOriginal;
cv::Mat matOriginalHSV;
cv::Mat matProcessed;
std::vector<cv::Vec3f> vecCircles;
std::vector<cv::Vec3f>::iterator itrCircles;
char charEscKey = 0;
cv::namedWindow("Original", CV_WINDOW_AUTOSIZE);
cv::namedWindow("Processed", CV_WINDOW_AUTOSIZE);
// creating trackbars
cv::namedWindow("Trackbar", CV_WINDOW_AUTOSIZE);
int hmin = 0, smin = 0, vmin = 0,
hmax = 180, smax = 256, vmax = 256;
cv::createTrackbar("H min:", "Trackbar", &hmin, hmax);
cv::createTrackbar("H max:", "Trackbar", &hmax, hmax);
cv::createTrackbar("S min:", "Trackbar", &smin, smax);
cv::createTrackbar("S max:", "Trackbar", &smax, smax);
cv::createTrackbar("V min:", "Trackbar", &vmin, vmax);
cv::createTrackbar("V max:", "Trackbar", &vmax, vmax);
// main function
while (charEscKey != 27) {
if (capWebcam.read(matOriginal) == NULL) {
std::cout << "Error: frame not readed from Cam\n\n";
break;
}
// converting to HSV
cv::cvtColor(matOriginal, matOriginalHSV, CV_BGR2HSV);
// spliting channels, changing any channel from "matOriginalHSVchannels[0/1/2]" to "matZero" to remove it
cv::Mat matZero;
cv::Mat matOriginalHS;
matZero = cv::Mat::zeros(cv::Size(matOriginalHSV.cols, matOriginalHSV.rows), CV_8UC1);
cv::vector<cv::Mat> matOriginalHSVchannels(3);
cv::split(matOriginalHSV, matOriginalHSVchannels);
cv::vector<cv::Mat> channels;
channels.push_back(matOriginalHSVchannels[0]);
channels.push_back(matOriginalHSVchannels[1]);
channels.push_back(matZero);
cv::merge(channels, matOriginalHS);
// search of laser spot
cv::inRange(matOriginalHS, cv::Scalar(hmin, smin, vmin), cv::Scalar(hmax, smax, vmax), matProcessed);
// blur
cv::GaussianBlur(matProcessed,
matProcessed,
cv::Size(5, 5),
2.2);
// circles search
cv::HoughCircles(matProcessed,
vecCircles,
CV_HOUGH_GRADIENT,
1,
matProcessed.rows / 8,
100,
50,
7,
20);
// draw circles
for (itrCircles = vecCircles.begin(); itrCircles != vecCircles.end(); itrCircles++) {
std::cout << "position x = " << (*itrCircles)[0]
<< ", y = " << (*itrCircles)[1]
<< ", z = " << (*itrCircles)[2] << "\n";
cv::circle(matOriginalHS,
cv::Point((int)(*itrCircles)[0], (int)(*itrCircles)[1]),
1,
cv::Scalar(255, 0, 0),
CV_FILLED);
cv::circle(matOriginalHS,
cv::Point((int)(*itrCircles)[0], (int)(*itrCircles)[1]),
(int)(*itrCircles)[2],
cv::Scalar(255, 255, 0),
2);
// set mouse cursor on laser coordinates
int screenx = (int)((*itrCircles)[0] * 2.2);
int screeny = (int)((*itrCircles)[1] * 1.2);
SetCursorPos(screenx, screeny);
}
// show windows
cv::imshow("Original", matOriginalHS);
cv::imshow("Processed", matProcessed);
charEscKey = cv::waitKey(10);
}
return(0);
}
It is not possible to detect the laser pointer with this code even at the minimal brightness of the projector. All the more it will be necessary to use the projector at the maximum brightness. I tried to use trackbar regulators, convert to different color schemes and delete separate layers.
The screenshots are attached. The first one shows the white text and laser separately. At the second the laser points to the letter "S" and it is not seen any more. If there are big white objects instead of the text, the webcam can't detect the laser at all. I tried to write algorithms to correct camera and white color balance, but it didn't help.
If you have any advice or hint of how to detect the red laser pointer at the dynamic projection with white elements, please help me.
