# Smooth point feature trajectories by Kalman Filter in Video Stabilization

Now, I am researching the topic Video Stabilization. I selected good point features (PFs) to track based on Kanade-Lucas-Tomasi (KLT) tracker as a point feature tracker. After extract N PFs from the first image and tracking the PFs in the next image, Point Feature Trajectories (PFTs) are updated by connecting the tracked N PFs to previous PFTs. And continue,

Now I had a set of PFTs. I want to smooth this set of PFTs to create smooth point feature trajectories (SPFTs) by Kalman Filter. But this SPFTs is seemly like as PFTs. I dont know how to adjust the Kalman Filter parameters. Please help me to find out. Thank you in advance.

//Declare Kalman Filter
KalmanFilter KF (4,2,0);
Mat_<float> state (4,1);
Mat_<float> measurement (2,1);

void init_kalman(double x, double y)
{

KF.statePre.at<float>(0) = x;
KF.statePre.at<float>(1) = y;
KF.statePre.at<float>(2) = 0;
KF.statePre.at<float>(3) = 0;

KF.transitionMatrix = *(Mat_<float>(4,4) << 1,0,1,0,    0,1,0,1,     0,0,1,0,   0,0,0,1);
KF.processNoiseCov = *(Mat_<float>(4,4) << 0.2,0,0.2,0,  0,0.2,0,0.2,  0,0,0.3,0,
0,0,0,0.3);
setIdentity(KF.measurementMatrix);
setIdentity(KF.processNoiseCov,Scalar::all(1e-4));
setIdentity(KF.measurementNoiseCov,Scalar::all(1e-1));
setIdentity(KF.errorCovPost, Scalar::all(.1));
}

Point2f kalman_predict_correct(double x, double y)
{
Mat prediction = KF.predict();
Point2f predictPt (prediction.at<float>(0), prediction.at<float>(1));
measurement(0) = x;
measurement(1) = y;
Mat estimated = KF.correct(measurement);
Point2f statePt (estimated.at<float>(0), estimated.at<float>(1));
return statePt;
}
// SMOOTH DATA is runned in main function
measurement.setTo(Scalar(0));
for(size_t m = 0; m < new_track_feature.size(); m++)
{
for(size_t n = 0; n < new_track_feature[0].point_list.size(); n++)
{
init_kalman(new_track_feature[m].point_list[n].point.x,
new_track_feature[m].point_list[n].point.y);
Point2f smooth_feature =
kalman_predict_correct(new_track_feature[m].point_list[n].point.x,
new_track_feature[m].point_list[n].point.y);
smooth_feature_point.push_back(PointFeature(n,smooth_feature,USE));
}
smooth_feature_track.push_back(TrackFeature(m,smooth_feature_point));
}

edit retag close merge delete

1

imho you want to use the predicted point, not the statePost of the filter (the return val from correct())

also, you might try even smaller values here :

setIdentity(KF.processNoiseCov,Scalar::all(1e-6));

( 2014-03-27 11:35:36 -0600 )edit

I did it as your advice. But the result is same. Hic

( 2014-03-28 03:08:18 -0600 )edit

oh, another thing : idk, how often you are calling that for-loop, but you should init your kalman filter(s) only once , not each time you're calling predict() or correct(). they have to preserve the state between 'frames', imho, you're destroying that.

( 2014-03-28 03:27:34 -0600 )edit

where is for loop?

( 2014-03-28 05:04:22 -0600 )edit

for(size_t m = 0; m < new_track_feature.size(); m++)

( 2014-03-28 05:22:02 -0600 )edit

Yes, just once for loop with m times

( 2014-03-28 05:32:19 -0600 )edit

ooh, now i see it. here's the main problem:

you need a separate kalman filter for each point/object you want to track.

and again, initialize it once, then call predict()/correct() for each new image

( 2014-03-28 05:50:10 -0600 )edit

Yes, let me try. Thank you so much.

( 2014-03-28 06:13:48 -0600 )edit

@Jenny , are you able to smooth the data ? What do you have in output of Kalman ? Because i would like to smooth the trajectory but there is something wrong...

( 2014-12-04 05:30:42 -0600 )edit