Ask Your Question
0

Remove jitter in video stabilization

asked 2014-02-06 23:34:39 -0600

Jenny gravatar image

updated 2014-02-06 23:35:09 -0600

I am working on a video stabilization field. Now I calculate optical flow between 2 consecutive frames by using calcOpticalFlowPyrLK in opencv. After that, I use function estimateGlobalMotionRobust to get global motion parameters to warp the previous frame to new frame. But the result is not good. it is not stable as I desired. I read many documents for this stabilize purpose is need to remove jitter in global motion parameters. But I do not know how to... Some documents said that I should use Kalman filter, another use low pass filter, but I do not know to how to use them. Can you help me?

Here is my code:

VideoWriter writeOutputVideo ("out.avi",0,fps,cvSize(frameW,frameH),TRUE);
for(;;)

{

capture.read(colorImg);

cvtColor(colorImg,grayImg,CV_BGR2GRAY);

prevImg = currImg.clone();

currImg = grayImg.clone();

backupColorImg = colorImg.clone();


vector<Point2f>cornerPrev;      

cornerPrev.reserve(maxCorners);

vector<Point2f>cornerCur;
cornerCur.reserve(maxCorners);

TermCriteria opticalFlowTermCriteria = TermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS,20,0.3);

goodFeaturesToTrack(prevImg,cornerPrev,maxCorners,qualityLevel,minDistance,Mat());
cornerSubPix(prevImg,cornerPrev,Size(winSize,winSize),Size(-1,-1),opticalFlowTermCriteria);


std::vector<uchar> features_found;
features_found.reserve(maxCorners);

std::vector<float> feature_errors;
feature_errors.reserve(maxCorners);

calcOpticalFlowPyrLK( prevImg, currImg, cornerPrev, cornerCur, features_found, feature_errors ,Size( winSize, winSize ), 5,opticalFlowTermCriteria, 0 );

//Mat transformMatrix = estimateGlobalMotionLeastSquares(cornerPrev,cornerCur,AFFINE,0);
//Mat transformMatrix = findHomography(cornerPrev,cornerCur,0,3.0,noArray());
Mat transformMatrix = estimateGlobalMotionRobust(cornerPrev,cornerCur,3,RansacParams::affine2dMotionStd(),0,0);

warpPerspective(backupColorImg,outImg,transformMatrix,Size(frameW,frameH),INTER_NEAREST|WARP_INVERSE_MAP,BORDER_CONSTANT ,0);


writeOutputVideo.write(outImg);

imshow("Input",colorImg);
imshow("Optical Flow",outImg);

}

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-02-07 02:55:47 -0600

I don't think that KLT is the best algorithm for video stabilization... It's way too slow...

edit flag offensive delete link more

Comments

Yes I see, but my purpose is be stable video, not care about the speed. Can you help me to fix the above problem, please?

Jenny gravatar imageJenny ( 2014-02-09 01:06:05 -0600 )edit
0

answered 2014-02-07 17:22:11 -0600

Nghia gravatar image

As a first step I would confirm that you are getting good optical flow by drawing them on the screen.

As a second step I would remove bad optical flow vectors before passing them into estimateGlobalMotionRobust. Just loop through features_found and keep the good ones. This may or may not not be necessary, but if you can make life easier for the function then why not.

edit flag offensive delete link more

Comments

I already checked the optical flow. It 's exactly. But when I get the global motion parameter through transformMatrix variable to warp image, then the result video is not stable. I do not know why it is? How do you know the bad optical flow vectors?

Jenny gravatar imageJenny ( 2014-02-09 01:05:09 -0600 )edit

features_found is a vector containing "1" for good, "0" for bad. Would you be able to post the input video you use? That way we can see what's going on.

Nghia gravatar imageNghia ( 2014-02-09 05:17:05 -0600 )edit

Yes. My input video code is here. I read a video with format .avi from the computer.

   Mat currImg, prevImg, colorImg, outImg, grayImg, resultImg, backupColorImg;
if (!capture.isOpened())
{
    printf("Can not open the video file");
    exit;
}
capture.read(colorImg);
cvtColor(colorImg,grayImg,CV_BGR2GRAY);
currImg = grayImg.clone();
outImg = grayImg.clone();



int winSize = 20;
int maxCorners = 200;

double qualityLevel = 0.01;
double minDistance = 5.0;
int blockSize = 3;

int fps = 30;
int frameW = grayImg.cols;
int frameH = grayImg.rows;
Jenny gravatar imageJenny ( 2014-02-09 05:40:54 -0600 )edit

What I meant was the video you use for testing, the file that you load in. Would you be able to upload the video file, or is too big?

Nghia gravatar imageNghia ( 2014-02-09 05:46:58 -0600 )edit

My video test: http://www.youtube.com/watch?v=FfXfQSpJAEw. I uploaded it temporally

Jenny gravatar imageJenny ( 2014-02-09 06:06:32 -0600 )edit

That's a very low res video, might be hard to work with. Do you have one that is a bit higher resolution?

Nghia gravatar imageNghia ( 2014-02-09 06:56:16 -0600 )edit
Jenny gravatar imageJenny ( 2014-02-09 07:05:01 -0600 )edit

Question Tools

Stats

Asked: 2014-02-06 23:34:39 -0600

Seen: 2,718 times

Last updated: Feb 07 '14