fitEllipse, least-squares error metric

edit savecancel

I'm interested in fitting an ellipse to a set of data points that contains a decent amount of noise. To do that, I'd like to implement the well known RANSAC algorithm. Now to do that, I need to compare the best fit least-squares value for each ellipse I'm fitting.

I have looked over the code implemented in cv::fitEllipse but failed to understand where such a value might be. Is the implementation similar to http://nicky.vanforeest.com/misc/fitEllipse/fitEllipse.html? If so, if I understood correctly, I need to return the eigenvector corresponding to the largest eigenvalue of the last equation.

I suspect the arrays gfp and rp to be of interest but I'm not sure at all.