Shape Transformers and Interfaces 2

asked 2015-10-02 06:03:20 -0500

Ives gravatar image

updated 2015-10-02 07:54:43 -0500

Hey, in this Shape Transformer Interface seems to be buggy. Maybe also see my former Question

The following code will only produce a completly gray image. The error is reproducable by giving the last point the same x/y value e.g. 2/2, but will work perfectly fine with 2/3. But thats only one possibility to cause this error. I found other picutres with "real" landmark points which also cause this error but I couldn't figure out a pattern. (Let me know if you need the data) I would be grateful for any help :)

int main(void)
cv::Mat sImg = cv::imread("D:\\Opencv3\\sources\\samples\\data\\graf1.png", cv::IMREAD_GRAYSCALE);
std::vector<cv::Point2f> points;
std::vector<cv::DMatch> good_matches;
cv::Mat tImg;

points.push_back(cv::Point(0, 0)); //Just to have a few points
points.push_back(cv::Point(1, 1));
points.push_back(cv::Point(2, 2)); // points.push_back(cv::Point(2, 3)) -> works fine

good_matches.push_back(cv::DMatch(0, 0, 0));
good_matches.push_back(cv::DMatch(1, 1, 0));
good_matches.push_back(cv::DMatch(2, 2, 0));

// Apply TPS
cv::Ptr<cv::ThinPlateSplineShapeTransformer> mytps = cv::createThinPlateSplineShapeTransformer(0);
mytps->estimateTransformation(points, points, good_matches); // Using same points nothing should change in tImg

imshow("sImg", sImg); // Just to see if I have a good picture

mytps->warpImage(sImg, tImg);

imshow("tImg", tImg); //Always completley gray ?

edit retag flag offensive close merge delete


Hello, although the solution of this particular example is trivial; the TPS is not really defined for control points that are colinear. This should be asserted in the estimateTransformation method. Could you look into this a send a PR?

juanmanpr gravatar imagejuanmanpr ( 2015-10-04 03:59:34 -0500 )edit

Ok you're right on using only collinear control points, but I can assure you that's not the only problem. Right now I can't check but I'm pretty sure it also failed when I set the first two points not collinear. And I can definitely tell that the TPS failed with 217 non trivial control points of face landmarks. (Same controllpoints woked perfectly fine in an Matlab implementation of TPS) I already started debugging the TPS but for the next two weeks its not priority.

Ives gravatar imageIves ( 2015-10-05 04:11:41 -0500 )edit

Hi there. I'm planning on working with the Opencv TPS function myself for a project and I was wondering whether you figured out what was the problem?

MrGiskard gravatar imageMrGiskard ( 2016-10-30 13:13:27 -0500 )edit