Smooth point feature trajectories by Kalman Filter in Video Stabilization
Now, I am researching the topic Video Stabilization. I selected good point features (PFs) to track based on Kanade-Lucas-Tomasi (KLT) tracker as a point feature tracker. After extract N PFs from the first image and tracking the PFs in the next image, Point Feature Trajectories (PFTs) are updated by connecting the tracked N PFs to previous PFTs. And continue,
Now I had a set of PFTs. I want to smooth this set of PFTs to create smooth point feature trajectories (SPFTs) by Kalman Filter. But this SPFTs is seemly like as PFTs. I dont know how to adjust the Kalman Filter parameters. Please help me to find out. Thank you in advance.
//Declare Kalman Filter
KalmanFilter KF (4,2,0);
Mat_<float> state (4,1);
Mat_<float> measurement (2,1);
void init_kalman(double x, double y)
{
KF.statePre.at<float>(0) = x;
KF.statePre.at<float>(1) = y;
KF.statePre.at<float>(2) = 0;
KF.statePre.at<float>(3) = 0;
KF.transitionMatrix = *(Mat_<float>(4,4) << 1,0,1,0, 0,1,0,1, 0,0,1,0, 0,0,0,1);
KF.processNoiseCov = *(Mat_<float>(4,4) << 0.2,0,0.2,0, 0,0.2,0,0.2, 0,0,0.3,0,
0,0,0,0.3);
setIdentity(KF.measurementMatrix);
setIdentity(KF.processNoiseCov,Scalar::all(1e-4));
setIdentity(KF.measurementNoiseCov,Scalar::all(1e-1));
setIdentity(KF.errorCovPost, Scalar::all(.1));
}
Point2f kalman_predict_correct(double x, double y)
{
Mat prediction = KF.predict();
Point2f predictPt (prediction.at<float>(0), prediction.at<float>(1));
measurement(0) = x;
measurement(1) = y;
Mat estimated = KF.correct(measurement);
Point2f statePt (estimated.at<float>(0), estimated.at<float>(1));
return statePt;
}
// SMOOTH DATA is runned in main function
measurement.setTo(Scalar(0));
for(size_t m = 0; m < new_track_feature.size(); m++)
{
for(size_t n = 0; n < new_track_feature[0].point_list.size(); n++)
{
init_kalman(new_track_feature[m].point_list[n].point.x,
new_track_feature[m].point_list[n].point.y);
Point2f smooth_feature =
kalman_predict_correct(new_track_feature[m].point_list[n].point.x,
new_track_feature[m].point_list[n].point.y);
smooth_feature_point.push_back(PointFeature(n,smooth_feature,USE));
}
smooth_feature_track.push_back(TrackFeature(m,smooth_feature_point));
}
imho you want to use the predicted point, not the statePost of the filter (the return val from correct())
also, you might try even smaller values here :
I did it as your advice. But the result is same. Hic
oh, another thing : idk, how often you are calling that for-loop, but you should init your kalman filter(s) only once , not each time you're calling predict() or correct(). they have to preserve the state between 'frames', imho, you're destroying that.
where is for loop?
for(size_t m = 0; m < new_track_feature.size(); m++)
Yes, just once for loop with m times
ooh, now i see it. here's the main problem:
you need a separate kalman filter for each point/object you want to track.
and again, initialize it once, then call predict()/correct() for each new image
Yes, let me try. Thank you so much.
@Jenny , are you able to smooth the data ? What do you have in output of Kalman ? Because i would like to smooth the trajectory but there is something wrong...