Problems with calcOpticalFlowPyrLK [closed]

asked 2016-11-14 10:40:59 -0500

Vintez gravatar image

I want to use the calcOpticalFlowPyrLK() to track Points I found after a (larger) detection/Matching of a Image. The first recognition works with FAST/SIFT and matches the found Points are stored for the OpticalFlow Tracking.

I tried to eliminate Points where the status is not 1. After that I want calculate the Homography of the old Point set from the recognition and the new Point set from tracking. Afterwards, I want to calculate the perspective Transformation on the Corner Points so I can Mark the detected Object with a rectangle.

Enough talk here the Code and my Output:

std::cout << "****KLT TRACKING ****" << std::endl;


cv::Mat firstImg = cv::imread(imgPathtest7, -1);
cv::Mat firstgry;

cvtColor(firstImg, firstgry, CV_BGR2GRAY);

std::vector<cv::Point2f> estPoints;
std::vector<uchar> stat;
std::vector<float> errs;

cv::calcOpticalFlowPyrLK(gry, firstgry, ransacs, estPoints, stat, errs);

std::vector<cv::Point2f> estCorners;

for(int i = 0; i < estPoints.size(); i++){
    if(stat[i]){
    cv::circle(firstImg, estPoints[i], 5, cv::Scalar(0, 0, 255), 2);
    std::cout << "Estimated Point:" << estPoints[i] << std::endl;
    std::cout << "Status: " << stat[i] << std::endl;
    } else {
        estPoints.erase(estPoints.begin() + i);
        ransacs.erase(ransacs.begin() + i);
    }
    std::cout << "Error: " << errs[i] << std::endl;
}

std::cout << "Size of ransacs/estPoints: " << ransacs.size() << " " << estPoints.size() << std::endl;

cv::Mat f = cv::findHomography(ransacs, estPoints);

std::cout << "Homo: " << f << std::endl;

cv::perspectiveTransform(scene_corners, estCorners, f);

cv::line(firstImg, estCorners[0], estCorners[1], cv::Scalar(0, 255, 0), 4);
cv::line(firstImg, estCorners[1], estCorners[2], cv::Scalar(0, 255, 0), 4);
cv::line(firstImg, estCorners[2], estCorners[3], cv::Scalar(0, 255, 0), 4);
cv::line(firstImg, estCorners[3], estCorners[0], cv::Scalar(0, 255, 0), 4);

cv::imshow("First Iteration", firstImg);

 std::cout << "**** NEXT KLT TRACK ****" << std::endl;

//Second Image
cv::Mat nextMatchImg = cv::imread(imgPathtest6, -1);//STRING)
cv::Mat nextGry;

cvtColor(nextMatchImg, nextGry, CV_BGR2GRAY);

std::vector<cv::Point2f> estimatedPoints;
std::vector<uchar> status;
std::vector<float> err;

cv::calcOpticalFlowPyrLK(firstgry, nextGry, estPoints, estimatedPoints, status, err);

std::vector<cv::Point2f> estimatedCorners;

for(int i = 0; i < estimatedPoints.size(); i++){
    if(status[i]){
    cv::circle(nextMatchImg, estimatedPoints[i], 5, cv::Scalar(0, 0, 255), 2);
    std::cout << "Estimated Point:" << estimatedPoints[i] << std::endl;
    std::cout << "Status: " << status[i] << std::endl;
    }else{
        estimatedPoints.erase(estimatedPoints.begin() + i);
        estPoints.erase(estPoints.begin() + i);
    }
    std::cout << "Error: " << err[i] << std::endl;
}

std::cout << "Size of ransacs/estimatedPoints: " << ransacs.size() << " " << estimatedPoints.size() << std::endl;

cv::Mat f2 = cv::findHomography(estPoints, estimatedPoints);

std::cout << "Homo: " << f2 << std::endl;


cv::perspectiveTransform(estCorners, estimatedCorners, f2);

cv::line(nextMatchImg, estimatedCorners[0], estimatedCorners[1], cv::Scalar(0, 255, 0), 4);
cv::line(nextMatchImg, estimatedCorners[1], estimatedCorners[2], cv::Scalar(0, 255, 0), 4);
cv::line(nextMatchImg, estimatedCorners[2], estimatedCorners[3], cv::Scalar(0, 255, 0), 4);
cv::line(nextMatchImg, estimatedCorners[3], estimatedCorners[0], cv::Scalar(0, 255, 0), 4);

cv::imshow("Example KLT", nextMatchImg);

After that I get this Result:

image description

The left Image is the Result after Recogniton. The next two Images ... (more)

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by Vintez
close date 2016-12-01 02:36:17.524977

Comments

1

Looked quickly the code: I am not sure about your code for erasing bad points. The index is not the same after one erase call. You should use another vector and add the good point.

Also, you could try to draw the lines for the optical flow between the previous / current images for debugging.

Eduardo gravatar imageEduardo ( 2016-11-14 14:08:36 -0500 )edit

Wow I thank you for mentioning that but I still can't filter bad ones through looking at the Status. When is a Point signed as an Outlier in that function? I have a Error at some Points of up to 120 and they pass this function as Inlier. When I instead choose to filter them by Error it works fine but I'm not quite sure which error ratio would be good there

Vintez gravatar imageVintez ( 2016-11-15 01:54:41 -0500 )edit