how to fit an ellipse to 2D data with random three points and thire gradient?

edit

i need to fit an ellipse to 2D data with random three points and their gradient. and the points is the edge points of one image. i use the Opencv function to calculate the gradient of each point.

i konw the RANSAC algorithm can do this job. and this algorithm can find the inner points fitting the ellipse and reject the noise points. but this algorithm uses five random points and it increases the sample times. so i want to improve this algorithm.

now considering the ellipse property, i find the three random points and their gradient can fit an ellipse. i have done this job combining the RANSAC algorithm. i choose three random points, calculate the ellipse center with their gradient. when i choose the fourth point and add some constraint of the ellipse. lastly, i find the inner points. but it doesnot work well.

so i want to know how to make this job well, especially restraining the fourth point with the ellipse property.

thanks! this is all the points

this is the result(the green points is the inner points, the red is the noise, the blue is the center)