MarKS9's profile - activity

@berak yes i agree handling GUI's in separate thread is dangerous. I have indeed compiled OpenCV with OpenMP, CUDA, OpenCL and TBB. I just wanted to accomplish this multiple frames capture for marker detection but it seems way too complex. i followed this well written article link text and i managed to implement it for small opencv programs but could'nt for marker detection.

@berak true! it is complex. But i have to start at some point. Thank you.

@Der Luftmensch thank you very much for your suggestion will check it out.

I have a marker detection code which runs fine with a video file and camera stream. I want to modify the code to grab frames from 2 videos in 2 different threads and then process marker detection accordingly. Here is a Psuedo code what i really want to do:

string camera[2] = {"test.avi","test1.avi"};
int main(){
VideoCapture cap();
while(1){
            int threads = get_num_threads(2);
            cap >> frame;
            marker_detection(frame);
            namedWindow(Camera[Thread], CV_WINDOW_NORMAL);
            imshow(Camera[Thread], frame);
            waitKey(1);
        }
}

It is just the pseudo code. I am completely new to multithreading. How do i grab a frame from both the files in parallel and then process marker detection for both the frames simultaneously? Note: i have gui functions like imshow() inside marker detection function.

I am new to Computer Vision world. I have implemented opencv SURF algorithm on PC and embedded boards. I am getting a decent fps output. I want to classify SURF algorithm according to performance on CPU, GPU, minHessian matrix and plot the output results for different boards. I already implemented it for CPU, GPU and different minHessian values. But i want to parallelize SURF code from inside to increase the performance. Anyone know how do i do that. I managed to find some interesting articles like

http://www.isaet.org/images/extraimag...

http://recsi2012.mondragon.edu/es/pro...

In my code i am tracking a QR code. Please help me with this. Thanks

thank you all for the response. I have finally solved it as per your suggestions

if (good_matches.size() >= 4){
line(img_matches, scene_corners[0] + Point2f(src.cols, 0), scene_corners[1] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[1] + Point2f(src.cols, 0), scene_corners[2] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[2] + Point2f(src.cols, 0), scene_corners[3] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[3] + Point2f(src.cols, 0), scene_corners[0] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
}

as per @StevenPuttemans suggestion i managed to reinitialize scene_vector inside the capture loop. And its working perfectly fine.

@StevenPuttemans i am calling surf detection using a function inside my capture loop. So yes, my scene_corners is outside my main(). How do i reinitialize scene_corners?

Well sorry for this stupid question. Thank you @Tetragramm & @berak it worked. But it is so slow to find the object in the new position. I mean the rectangle doesn't follow the object in the scene in real time and it takes around 8-9 secs to draw new rectangle around the object. I want it like in this video https://www.youtube.com/watch?v=ZXn69.... Any idea how do i do that?

Please check the edited question.

I am using SURF for detecting specific objects. But the issue with my code is the rectangle is drawn even if the object is not present in the scene. Like if i remove the object from the scene the rectangle stays there. I have no issues with detection it works perfectly.

//Ptr<SIFT> detector = SIFT::create(0,5,0.03,18);
Ptr<SURF> detector = SURF::create(minHessian,4,2);
Ptr<DescriptorExtractor> extractor = SURF::create(minHessian,4,2);
//Ptr<DescriptorExtractor> extractor = SIFT::create(0,5,0.03,18);

cvtColor(dst, src, CV_BGR2GRAY);
cvtColor(frame, gray_image, CV_BGR2GRAY);


detector->detect(src, keypoints_1);
//printf("Object: %d keypoints detected\n", (int)keypoints_1.size());
detector->detect(gray_image, keypoints_2);
//printf("Object: %d keypoints detected\n", (int)keypoints_1.size());

extractor->compute(src, keypoints_1, img_extractor);
// printf("Object: %d descriptors extracted\n", img_extractor.rows);
extractor->compute(gray_image, keypoints_2, snap_extractor);

std::vector<Point2f> scene_corners(4);
std::vector<Point2f> obj_corners(4);

obj_corners[0] = (cvPoint(0, 0));
obj_corners[1] = (cvPoint(src.cols, 0));
obj_corners[2] = (cvPoint(src.cols, src.rows));
obj_corners[3] = (cvPoint(0, src.rows));

vector<DMatch> matches;
matcher.match(img_extractor, snap_extractor, matches);

double max_dist = 0; double min_dist = 100;

//-- Quick calculation of max and min distances between keypoints
for (int i = 0; i < img_extractor.rows; i++)
{
    double dist = matches[i].distance;
    if (dist < min_dist) min_dist = dist;
    if (dist > max_dist) max_dist = dist;
}
printf("-- Max dist : %f \n", max_dist);
printf("-- Min dist : %f \n", min_dist);

vector< DMatch > good_matches;

for (int i = 0; i < img_extractor.rows; i++)
{
    if (matches[i].distance <= max(2 * min_dist, 0.02))
    {
        good_matches.push_back(matches[i]);
    }
}

Mat img_matches;
vector< DMatch > emptyVec;
drawMatches(src, keypoints_1, gray_image, keypoints_2, emptyVec,
    img_matches, Scalar::all(-1), Scalar::all(-1),
    vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);

if (good_matches.size() >= 4){

    for (size_t i = 0; i<good_matches.size(); i++){

        //get the keypoints from good matches
        obj.push_back(keypoints_1[good_matches[i].queryIdx].pt);
        scene.push_back(keypoints_2[good_matches[i].trainIdx].pt);

    }
}

H = findHomography(obj, scene, CV_RANSAC);

perspectiveTransform(obj_corners, scene_corners, H);

line(img_matches, scene_corners[0] + Point2f(src.cols, 0), scene_corners[1] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[1] + Point2f(src.cols, 0), scene_corners[2] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[2] + Point2f(src.cols, 0), scene_corners[3] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[3] + Point2f(src.cols, 0), scene_corners[0] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);

I want to draw rectangle only when the object in the scene is detected. How do i do it?

thank you very much! It worked. I am not sure how did i miss this link.

I have an optical flow algorithm which uses Updatemotionhistory but it runs on CPU. I want to check the performance of this algorithm on GPU. So i plan to use UMat. Here is a snippet

    UMat gpu_motion_mask,gpu_motion_history;
    motion_mask.copyTo(gpu_motion_mask);
    motion_history.copyTo(gpu_motion_history);
    updateMotionHistory(gpu_motion_mask, gpu_motion_history, timestamp, MHI_DURATION);
    calcMotionGradient(gpu_motion_history, mg_mask, mg_orient, MIN_TIME_DELTA, MAX_TIME_DELTA, 3);
    segmentMotion(gpu_motion_history, seg_mask, seg_bounds, timestamp, 32);

In the above code i converted Mat into UMat to use it in the functions. But after this i need to access the pixels in the UMat gpu_motion_history and gpu_motion_mask using at<float>.

    for(int i=0; i< motion_history.cols; i++)
            {
                for(int j=0; j< motion_history.rows ; j++)
                {
                float a = motion_history.at<float>(j,i);
              //                    cout << (a-timestamp-MHI_DURATION)/MHI_DURATION << endl;
                if((a-timestamp-MHI_DURATION)/MHI_DURATION <= -5)
                    vis1.at<uchar>(j,i) = 0;
                else
                    vis1.at<uchar>(j,i) = (a-timestamp-MHI_DURATION)/MHI_DURATION;
            }

Issue is it thorws an error saying cv::UMat has no member named "at". Is there any way i can access the pixels in UMat?

Size of the image is 320X640. Comparing to CPU it comes to a crawl.
CPU - 7-8 fps GPU - 1 fps

I have implemented SURF with CUDA but when i execute it runs dead slow and my mouse freezes intermittently while it is running. I am not able to figure out is it hardware issue or there something wrong in my code.

void surf_detect::start_detect()
{
// detecting keypoints & computing descriptors

vector<Point2f> scene_corners(4);
vector<Point2f> obj_corners(4);

surf(img1, GpuMat(), keypoints1GPU, descriptors1GPU);
surf(img2, GpuMat(), keypoints2GPU, descriptors2GPU);

cout << "FOUND " << keypoints1GPU.cols << " keypoints on first image" << endl;
cout << "FOUND " << keypoints2GPU.cols << " keypoints on second image" << endl;

// matching descriptors
Ptr<cv::cuda::DescriptorMatcher> matcher = cv::cuda::DescriptorMatcher::createBFMatcher(surf.defaultNorm());
vector<DMatch> matches;
matcher->match(descriptors1GPU, descriptors2GPU, matches);

// downloading results
vector<KeyPoint> keypoints1, keypoints2;
vector<float> descriptors1, descriptors2;
surf.downloadKeypoints(keypoints1GPU, keypoints1);
surf.downloadKeypoints(keypoints2GPU, keypoints2);
surf.downloadDescriptors(descriptors1GPU, descriptors1);
surf.downloadDescriptors(descriptors2GPU, descriptors2);

// drawing the results
vector<DMatch> emptyvec;
drawMatches(Mat(img1), keypoints1, Mat(img2), keypoints2, emptyvec, img_matches,Scalar::all(-1), Scalar::all(-1),
        vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);

line(img_matches, scene_corners[0] + Point2f(img1.cols, 0), scene_corners[1] + Point2f(img1.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[1] + Point2f(img1.cols, 0), scene_corners[2] + Point2f(img1.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[2] + Point2f(img1.cols, 0), scene_corners[3] + Point2f(img1.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[3] + Point2f(img1.cols, 0), scene_corners[0] + Point2f(img1.cols, 0), Scalar(0, 255, 0), 4);

namedWindow("matches", 0);
imshow("matches", img_matches);
waitKey(30);
}

I have a NVidia 520M graphics card with CUDA support. My sample cuda programs are running absolutely fine. Why is it so slow?

I have 2 video(.avi) files and i have successfully able to grab multiple frames using C++ threading and store it in frame_queue.

vector<concurrent_queue<Mat>*> frame_queue;

But i want to run marker detection on these captured frames.

Here is the code snippet

while (1)
{
    //Retrieve frames from each camera capture thread
    for (int i = 0; i < capture_source.size(); i++)
    {
        Mat vdo_frame;
        //Pop frame from queue and check if the frame is valid
        if (cam.frame_queue[i]->try_pop(vdo_frame)){
            //Show frame on Highgui window
            marker_detection(vdo_frame);
            //imshow(label[i], frame);
        }
    }
}

If i am not wrong cam.frame_queue[0] holds the frames from video1 and frame_queue[1] holds the frames from video2. But how do i run marker detection for both the videos in parallel? Issue is whenever i run the code it works for some seconds but then my Ubuntu freezes and i have to reboot.

Does your GPU support CUDA ? Did you install CUDA tool kit and test CUDA sample programs like deviceQuery?

I have successfully executed different opencv algorithms like Marker detection, SURF, Optical flow etc. I ported the applications on to ARM as well. Now i want to analyze the performance of these algorithms on both PC and embedded boards. How can i do benchmarking on the basis of performance for both PC and embedded boards?

well thank you for response. I figured out how to use concurrent_queue.h by installing TBB. I know this is not the answer for how to do it with vector<mat>. But i will try your suggestions for sure.

Sorry it was a typo. But i got to know that there is pop_back(). i can write cam.frame_queue.push_back() but not cam.frame_queue[i].push_back(). I am not sure why.

please check the edited question.

I have a vector frame_queue which i am using to store incoming frames from a USB camera. I want to pop out one single frame at a time to further process those frames. I followed this link https://putuyuwono.wordpress.com/2015...

The issue is i am using 2 USB cameras and i want to grab both frames in frame_queue vector. But i am not able to use this code to push

frame_queue[i].pushback(frame);

Here is a sample code i am using :

//Make an instance of CameraStreamer
CameraStreamer cam(capture_index);

while (waitKey(20) != 27)
{
    //Retrieve frames from each camera capture thread
    for (size_t i = 0; i < capture_index.size(); i++)
    {
        Mat frame;
        //Pop frame from queue and check if the frame is valid
        cam.frame_queue[i].push_back(frame);
        //if (cam.frame_queue[i]->try_pop(frame)){
        //Show frame on Highgui window
            imshow(label[i], frame);
        //}
    }
}

frame_queue is declared as vector<mat> frame_queue. And how to pop a frame from the frame_queue if i successfully push frames into it? How do i push and pop frames into and from frame_queue[i] ?

You need to install CUDA toolkit SDK and then try running a CUDA sample like deviceQuery if it runs fine then your CUDA is configured correctly.

i don't know what is the error but it happened to me once but i switched to version 2.4.9 then it worked. It is not an answer but you can give it a try.

Thank you! Could you please post some links for a sample code. Tried to google it but in vain.

I have implemented Optical flow on a video in which a group of people are running. I need to cluster them into a group and then track them accordingly.

I am confused which clustering algorithm should i be using K-means or Mean shift? Can anyone please help me out.

its a linker error. So try cross-checking if you have linked all the the valid libraries you are using in the project. If not, then try building OpenCV for 32-bit may be. this is not a solution but i did face problems with 64-bit libraries before so just a suggestion.

Did you build OpenCV WITH_CUDA=ON?

I want to detect a normal marker (Not an ArUco marker) using ArUco module in OpenCV3.0 . Below is the image for a normal marker.

I have implemented the ArUco detection but not able to figure out how to make ArUco module to detect custom markers.

aruco::DetectorParameters detectorParams;
if (parser.has("dp")) {
bool readOk = readDetectorParameters(parser.get<string>("dp"), detectorParams);
if (!readOk) {
    cerr << "Invalid detector parameters file" << endl;
    return 0;
}
}

aruco::Dictionary dictionary =
aruco::getPredefinedDictionary(aruco::PREDEFINED_DICTIONARY_NAME(dictionaryId));

Mat camMatrix, distCoeffs;
if (estimatePose) {
bool readOk = readCameraParameters(parser.get<string>("c"), camMatrix, distCoeffs);
if (!readOk) {
    cerr << "Invalid camera file" << endl;
    return 0;
}
}

// detect markers and estimate pose
aruco::detectMarkers(image, dictionary, corners, ids, detectorParams, rejected);
if (estimatePose && ids.size() > 0)
    aruco::estimatePoseSingleMarkers(corners, markerLength, camMatrix, distCoeffs, rvecs,
        tvecs);

// draw results
image.copyTo(imageCopy);
if (ids.size() > 0) {
    aruco::drawDetectedMarkers(imageCopy, corners, ids);

    if (estimatePose) {
        for (unsigned int i = 0; i < ids.size(); i++)
            aruco::drawAxis(imageCopy, camMatrix, distCoeffs, rvecs[i], tvecs[i],
                markerLength * 0.5f);
    }
}

if (showRejected && rejected.size() > 0)
    aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255));

imshow("out", imageCopy);
char key = (char)waitKey(waitTime);
if (key == 27) break;
}

I need to make this SURF detection code run in parallel to gain faster fps. Now with this code i am getting max of 4 fps. what is the best method i could make it faster by running it in parallel using parallel_for_ , multithreading ? If yes, how do i do it ? Need help badly! Below is my implemented code.

void surf_detection::surf_detect(){

Mat img_extractor, snap_extractor;

if (crop_image_.empty())
    cv_snapshot.copyTo(dst);
else
    crop_image_.copyTo(dst);
//dst = QImagetocv(crop_image_);

imshow("dst", dst);

Ptr<SURF> detector = SURF::create(minHessian);
Ptr<DescriptorExtractor> extractor = SURF::create(minHessian);

cvtColor(dst, src, CV_BGR2GRAY);
cvtColor(frame, gray_image, CV_BGR2GRAY);


detector->detect(src, keypoints_1);
//printf("Object: %d keypoints detected\n", (int)keypoints_1.size());
detector->detect(gray_image, keypoints_2);
//printf("Object: %d keypoints detected\n", (int)keypoints_1.size());

extractor->compute(src, keypoints_1, img_extractor);
// printf("Object: %d descriptors extracted\n", img_extractor.rows);
extractor->compute(gray_image, keypoints_2, snap_extractor);

std::vector<Point2f> scene_corners(4);
std::vector<Point2f> obj_corners(4);

obj_corners[0] = (cvPoint(0, 0));
obj_corners[1] = (cvPoint(src.cols, 0));
obj_corners[2] = (cvPoint(src.cols, src.rows));
obj_corners[3] = (cvPoint(0, src.rows));

vector<DMatch> matches;
matcher.match(img_extractor, snap_extractor, matches);

double max_dist = 0; double min_dist = 100;

//-- Quick calculation of max and min distances between keypoints
for (int i = 0; i < img_extractor.rows; i++)
{
    double dist = matches[i].distance;
    if (dist < min_dist) min_dist = dist;
    if (dist > max_dist) max_dist = dist;
}
//printf("-- Max dist : %f \n", max_dist);
//printf("-- Min dist : %f \n", min_dist);

vector< DMatch > good_matches;

for (int i = 0; i < img_extractor.rows; i++)
{
    if (matches[i].distance <= max(2 * min_dist, 0.02))
    {
        good_matches.push_back(matches[i]);
    }
}

Mat img_matches;
drawMatches(src, keypoints_1, gray_image, keypoints_2,
    good_matches, img_matches, Scalar::all(-1), Scalar::all(-1),
    vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);

if (good_matches.size() >= 4){

    for (int i = 0; i<good_matches.size(); i++){

        //get the keypoints from good matches
        obj.push_back(keypoints_1[good_matches[i].queryIdx].pt);
        scene.push_back(keypoints_2[good_matches[i].trainIdx].pt);

    }
}

H = findHomography(obj, scene, CV_RANSAC);

perspectiveTransform(obj_corners, scene_corners, H);

line(img_matches, scene_corners[0] + Point2f(src.cols, 0), scene_corners[1] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[1] + Point2f(src.cols, 0), scene_corners[2] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[2] + Point2f(src.cols, 0), scene_corners[3] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);
line(img_matches, scene_corners[3] + Point2f(src.cols, 0), scene_corners[0] + Point2f(src.cols, 0), Scalar(0, 255, 0), 4);

//    drawKeypoints(src,keypoints_1,img_keypoint1,Scalar::all(-1),DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);
//    drawKeypoints(gray_image,keypoints_2,img_keypoint2,Scalar::all(-1),DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);

//    QImage img_1 = cvToQImage(img_keypoint1);
//    QImage img_2 = cvToQImage(img_keypoint2);

imshow("Good matches", img_matches);
//    ui->label_2->setScaledContents(true);
//    ui->label->setPixmap(QPixmap::fromImage(img_2));
//    ui->label_2->setPixmap(QPixmap::fromImage(img_1));