Smooth point feature trajectories by Kalman Filter in Video Stabilization

asked 2014-03-27 10:41:21 -0500

Jenny gravatar image

Now, I am researching the topic Video Stabilization. I selected good point features (PFs) to track based on Kanade-Lucas-Tomasi (KLT) tracker as a point feature tracker. After extract N PFs from the first image and tracking the PFs in the next image, Point Feature Trajectories (PFTs) are updated by connecting the tracked N PFs to previous PFTs. And continue,

Now I had a set of PFTs. I want to smooth this set of PFTs to create smooth point feature trajectories (SPFTs) by Kalman Filter. But this SPFTs is seemly like as PFTs. I dont know how to adjust the Kalman Filter parameters. Please help me to find out. Thank you in advance.

//Declare Kalman Filter
 KalmanFilter KF (4,2,0);
 Mat_<float> state (4,1); 
 Mat_<float> measurement (2,1);

void init_kalman(double x, double y)
{

     KF.statePre.at<float>(0) = x;
     KF.statePre.at<float>(1) = y;
     KF.statePre.at<float>(2) = 0;
     KF.statePre.at<float>(3) = 0;

     KF.transitionMatrix = *(Mat_<float>(4,4) << 1,0,1,0,    0,1,0,1,     0,0,1,0,   0,0,0,1);
     KF.processNoiseCov = *(Mat_<float>(4,4) << 0.2,0,0.2,0,  0,0.2,0,0.2,  0,0,0.3,0,   
                                                                                  0,0,0,0.3);
     setIdentity(KF.measurementMatrix);
     setIdentity(KF.processNoiseCov,Scalar::all(1e-4));
     setIdentity(KF.measurementNoiseCov,Scalar::all(1e-1));
     setIdentity(KF.errorCovPost, Scalar::all(.1));  
}

Point2f kalman_predict_correct(double x, double y)
{
     Mat prediction = KF.predict();
     Point2f predictPt (prediction.at<float>(0), prediction.at<float>(1));   
     measurement(0) = x;
     measurement(1) = y; 
     Mat estimated = KF.correct(measurement);
     Point2f statePt (estimated.at<float>(0), estimated.at<float>(1));
     return statePt;
}
// SMOOTH DATA is runned in main function
measurement.setTo(Scalar(0));
for(size_t m = 0; m < new_track_feature.size(); m++)
{
    for(size_t n = 0; n < new_track_feature[0].point_list.size(); n++)
    {
         init_kalman(new_track_feature[m].point_list[n].point.x,                    
                                                 new_track_feature[m].point_list[n].point.y);
          Point2f smooth_feature = 
                            kalman_predict_correct(new_track_feature[m].point_list[n].point.x,             
                                                   new_track_feature[m].point_list[n].point.y);
          smooth_feature_point.push_back(PointFeature(n,smooth_feature,USE));
    }
    smooth_feature_track.push_back(TrackFeature(m,smooth_feature_point));
}
edit retag flag offensive close merge delete

Comments

1

imho you want to use the predicted point, not the statePost of the filter (the return val from correct())

also, you might try even smaller values here :

setIdentity(KF.processNoiseCov,Scalar::all(1e-6));
berak gravatar imageberak ( 2014-03-27 11:35:36 -0500 )edit

I did it as your advice. But the result is same. Hic

Jenny gravatar imageJenny ( 2014-03-28 03:08:18 -0500 )edit

oh, another thing : idk, how often you are calling that for-loop, but you should init your kalman filter(s) only once , not each time you're calling predict() or correct(). they have to preserve the state between 'frames', imho, you're destroying that.

berak gravatar imageberak ( 2014-03-28 03:27:34 -0500 )edit

where is for loop?

Jenny gravatar imageJenny ( 2014-03-28 05:04:22 -0500 )edit

for(size_t m = 0; m < new_track_feature.size(); m++)

berak gravatar imageberak ( 2014-03-28 05:22:02 -0500 )edit

Yes, just once for loop with m times

Jenny gravatar imageJenny ( 2014-03-28 05:32:19 -0500 )edit

ooh, now i see it. here's the main problem:

you need a separate kalman filter for each point/object you want to track.

and again, initialize it once, then call predict()/correct() for each new image

berak gravatar imageberak ( 2014-03-28 05:50:10 -0500 )edit

Yes, let me try. Thank you so much.

Jenny gravatar imageJenny ( 2014-03-28 06:13:48 -0500 )edit

@Jenny , are you able to smooth the data ? What do you have in output of Kalman ? Because i would like to smooth the trajectory but there is something wrong...

lilouch gravatar imagelilouch ( 2014-12-04 05:30:42 -0500 )edit