Smooth point feature trajectories by Kalman Filter in Video Stabilization

asked 2014-03-27 10:41:21 -0500

Jenny gravatar image

Now, I am researching the topic Video Stabilization. I selected good point features (PFs) to track based on Kanade-Lucas-Tomasi (KLT) tracker as a point feature tracker. After extract N PFs from the first image and tracking the PFs in the next image, Point Feature Trajectories (PFTs) are updated by connecting the tracked N PFs to previous PFTs. And continue,

Now I had a set of PFTs. I want to smooth this set of PFTs to create smooth point feature trajectories (SPFTs) by Kalman Filter. But this SPFTs is seemly like as PFTs. I dont know how to adjust the Kalman Filter parameters. Please help me to find out. Thank you in advance.

//Declare Kalman Filter
 KalmanFilter KF (4,2,0);
 Mat_<float> state (4,1); 
 Mat_<float> measurement (2,1);

void init_kalman(double x, double y)
{<float>(0) = x;<float>(1) = y;<float>(2) = 0;<float>(3) = 0;

     KF.transitionMatrix = *(Mat_<float>(4,4) << 1,0,1,0,    0,1,0,1,     0,0,1,0,   0,0,0,1);
     KF.processNoiseCov = *(Mat_<float>(4,4) << 0.2,0,0.2,0,  0,0.2,0,0.2,  0,0,0.3,0,   
     setIdentity(KF.errorCovPost, Scalar::all(.1));  

Point2f kalman_predict_correct(double x, double y)
     Mat prediction = KF.predict();
     Point2f predictPt (<float>(0),<float>(1));   
     measurement(0) = x;
     measurement(1) = y; 
     Mat estimated = KF.correct(measurement);
     Point2f statePt (<float>(0),<float>(1));
     return statePt;
// SMOOTH DATA is runned in main function
for(size_t m = 0; m < new_track_feature.size(); m++)
    for(size_t n = 0; n < new_track_feature[0].point_list.size(); n++)
          Point2f smooth_feature = 
edit retag flag offensive close merge delete



imho you want to use the predicted point, not the statePost of the filter (the return val from correct())

also, you might try even smaller values here :

berak gravatar imageberak ( 2014-03-27 11:35:36 -0500 )edit

I did it as your advice. But the result is same. Hic

Jenny gravatar imageJenny ( 2014-03-28 03:08:18 -0500 )edit

oh, another thing : idk, how often you are calling that for-loop, but you should init your kalman filter(s) only once , not each time you're calling predict() or correct(). they have to preserve the state between 'frames', imho, you're destroying that.

berak gravatar imageberak ( 2014-03-28 03:27:34 -0500 )edit

where is for loop?

Jenny gravatar imageJenny ( 2014-03-28 05:04:22 -0500 )edit

for(size_t m = 0; m < new_track_feature.size(); m++)

berak gravatar imageberak ( 2014-03-28 05:22:02 -0500 )edit

Yes, just once for loop with m times

Jenny gravatar imageJenny ( 2014-03-28 05:32:19 -0500 )edit

ooh, now i see it. here's the main problem:

you need a separate kalman filter for each point/object you want to track.

and again, initialize it once, then call predict()/correct() for each new image

berak gravatar imageberak ( 2014-03-28 05:50:10 -0500 )edit

Yes, let me try. Thank you so much.

Jenny gravatar imageJenny ( 2014-03-28 06:13:48 -0500 )edit

@Jenny , are you able to smooth the data ? What do you have in output of Kalman ? Because i would like to smooth the trajectory but there is something wrong...

lilouch gravatar imagelilouch ( 2014-12-04 05:30:42 -0500 )edit