Problem with estimateRigidTransform: mat dst is empty

asked 2017-06-06 03:26:44 -0500

JoeBroesel gravatar image

Hello everyone, I'm new to OpenCV, so it could be that it's just a understanding problem with the estimateRigidTransformation function:

In the following code i find the contours of two rigid translated objects in img1 and 2, but estimateRigidTransformation seems not to work like i thought it would. It would be nice if someone has an idea why the mat dst keeps empty. Thank you!

#include <iostream>
#include <string> 
#include <opencv2/highgui/highgui.
#include <opencv2/video/tracking.hpp>
//Function from to extract Contours
static std::vector<cv::Point> sampleContour( const cv::Mat& image, int n=300 )
   std::vector<std::vector<cv::Point>> contours;
   std::vector<cv::Point> all_points;
   cv::findContours(image, contours, cv::RETR_LIST, cv::CHAIN_APPROX_NONE);
   for (size_t i=0; i <contours.size(); i++)
       for (size_t j=0; j<contours[i].size(); j++)

   // In case actual number of points is less than n
   int dummy=0;
   for (int add=(int)all_points.size(); add<n; add++)
   // Uniformly sampling
   std::random_shuffle(all_points.begin(), all_points.end());
   std::vector<cv::Point> sampled;
   for (int i=0; i<n; i++)
   return sampled;

int main(){
// image reading
cv::Mat templateImage = cv::imread("1.jpg", cv::IMREAD_GRAYSCALE);
cv::Mat queryImage = cv::imread("2.jpg", cv::IMREAD_GRAYSCALE);

// contour extraction
std::vector<cv::Point> queryPoints, templatePoints;
queryPoints = sampleContour(queryImage);
templatePoints = sampleContour(templateImage);

// cast to vector<point2f>
std::vector<cv::Point2f> queryPoints2f, templatePoints2f;
cv::Mat(queryPoints).convertTo(queryPoints2f, cv::Mat(queryPoints2f).type());
cv::Mat(templatePoints).convertTo(templatePoints2f, cv::Mat(templatePoints2f).type());

cv::Mat R = cv::estimateRigidTransform(templatePoints2f,queryPoints2f,false);
std::cout <<"R:"  << R << std::endl; // R -> empty

 * Solution from
 * let the program crash

cv::Mat H = cv::Mat(3,3,R.type());<double>(0,0) =<double>(0,0);<double>(0,1) =<double>(0,1);<double>(0,2) =<double>(0,2);<double>(1,0) =<double>(1,0);<double>(1,1) =<double>(1,1);<double>(1,2) =<double>(1,2);<double>(2,0) = 0.0;<double>(2,1) = 0.0;<double>(2,2) = 1.0;

std::vector<cv::Point2f> result;

for(unsigned int i=0; i<result.size(); ++i)
    std::cout << result[i] << std::endl;
 return 0;
edit retag flag offensive close merge delete


I think if you give two contours templatePoints2f,queryPoints2f it means that templatePoints2f[n] must be matched with queryPoints2f [n] Algorithm uses ransac some matching errors are possible

LBerger gravatar imageLBerger ( 2017-06-06 04:03:23 -0500 )edit

Thank you for your response, if i get you right the order of the Points in the vector is relevant?

I noticed that if I canged the line to

    cv::Mat R = cv::estimateRigidTransform(templatePoints2f,templatePoints2f,false);


    cv::Mat R = cv::estimateRigidTransform(queryPoints2f,queryPoints2f,false);

i get an output in the form of something like this: R:[1, -6.938148514913645e-16, 7.614916766799279e-14; 6.938148514913645e-16, 1, -1.285599231237722e-13]

instead of this: R:[]

Nevertheless the points are random_shuffled, aren't they? Would it help if i upload the used pictures (rotated apple-shapes from MPEG-dataset)?

JoeBroesel gravatar imageJoeBroesel ( 2017-06-06 04:22:39 -0500 )edit

random_shuffled in a set = if you chose index i in first set in second set you chose index i too. I will test on basic sample.

LBerger gravatar imageLBerger ( 2017-06-06 04:29:25 -0500 )edit

try this :

    vector<Point>  vertex1;
    vector<Point>  vertex2;

    vertex1.push_back(Point(300, 300));
    vertex1.push_back(Point(500, 300));
    vertex1.push_back(Point(500, 500));
    vertex1.push_back(Point(300, 500));
    for (int i = 0; i < vertex1.size(); i++)
        vertex2.push_back(vertex1[i] + Point(200, 200));
    cout<< estimateRigidTransform(vertex1, vertex2, true)<<"\n";
    for (int i = 0; i < vertex1.size(); i++)
        vertex2.push_back(vertex1[vertex1.size()-i-1] + Point(200, 200));
    cout << estimateRigidTransform(vertex1, vertex2, true)<<"\n";
LBerger gravatar imageLBerger ( 2017-06-06 04:37:11 -0500 )edit

Thanks again! If I edit your dst Mat like described here, i was able to map the points in your example to the right corespondence with


which means I am still not 100% sure if the order in vector matters -but it seems to be unstable or luck. Anyhow, if i push more than 5 points in vertex1 dst is getting empty again, even if the bool fullAffine is set to true. I think that the function is limeted to a few points. if there is not other way to estimate a rigid transform in opencv with a lot of unsorted points,can i use findHomography (perhaps as overkill)?

JoeBroesel gravatar imageJoeBroesel ( 2017-06-06 07:53:53 -0500 )edit

May be you can try this.

LBerger gravatar imageLBerger ( 2017-06-06 08:47:04 -0500 )edit

Thank you, i will have a look. As a last question: is the order of points relevant for findHomography?

JoeBroesel gravatar imageJoeBroesel ( 2017-06-06 13:25:51 -0500 )edit
LBerger gravatar imageLBerger ( 2017-06-06 13:32:51 -0500 )edit

thanks for your code! really impressive performance, but it will take a while until I understand everything in detail!

JoeBroesel gravatar imageJoeBroesel ( 2017-06-07 13:16:35 -0500 )edit

A PR is in progress

LBerger gravatar imageLBerger ( 2017-06-07 13:24:11 -0500 )edit