Ask Your Question
1

Video Stabilization using opencv

asked 2014-02-12 12:27:51 -0500

Jenny gravatar image

I 'm researching about Video Stabilization field. I implement a application using OpenCV.

My progress such as:

Surf points extraction

Matching

estimateRigidTransform

warpAffine

But the result video is not be stable. Can anyone help me this problem or provide me some source code link to improve?

Here is my code:

VideoCapture cap ("test.avi");  

Mat currImg, colorImg, outImg, grayImg, backupColorImg;
int winSize = 20;
int maxCorners = 200;

double qualityLevel = 0.01;
double minDistance = 5.0;
int blockSize = 3;

int frameW, frameH;

cap.read(colorImg);
cvtColor(colorImg,grayImg,CV_BGR2GRAY);
currImg = grayImg.clone();

outImg = colorImg.clone();
int fps = 25;
frameW = grayImg.cols;
frameH = grayImg.rows;

Mat ref;
cap.read(ref);
cvtColor(ref,ref,CV_BGR2GRAY);

SurfDescriptorExtractor extractor;

int minHessian = 400;
vector<KeyPoint> keyPointRef;
SurfFeatureDetector detector(minHessian);
Mat descriptorRef;


detector.detect(ref(Range(0,ref.rows), Range::all()), keyPointRef);
extractor.compute(ref, keyPointRef, descriptorRef);

VideoWriter writeVideo ("result.avi",0,fps,cvSize(frameW,frameH),false);

namedWindow("Stabilize", 0);
namedWindow("GoodMatches", 0);

while(1)
{              
bool bScuccess = cap.read(colorImg);

if (!bScuccess)
{
    cout<< "Cannot read the frame form video file";
    break;
}

cvtColor(colorImg,grayImg,CV_BGR2GRAY);
currImg = grayImg.clone();
backupColorImg = colorImg.clone();

vector<KeyPoint> keyPointCurr;
Mat descriptorCurr;

detector.detect(currImg,keyPointCurr);
extractor.compute(currImg,keyPointCurr,descriptorCurr);

FlannBasedMatcher matcher;
vector<DMatch> matches;
matcher.match(descriptorCurr,descriptorRef,matches);

double maxDist = 0, minDist = 100;
for(int i = 0; i < descriptorCurr.rows; i++)
{
    double dist = matches[i].distance;

    if(dist < minDist) minDist = dist;
    if(dist > maxDist) maxDist = dist;
}

vector<DMatch>good_matches;
for (int i = 0; i < descriptorCurr.rows; i++)
{
    //if (matches[i].distance <= max(2*minDist, 0.02))
    if(matches[i].distance < 0.2)
    {
        good_matches.push_back(matches[i]);
    }
}

vector<Point2f>curPoint;
vector<Point2f>refPoint;

for (int i = 0; i < good_matches.size(); i++)
{
    curPoint.push_back(keyPointCurr[good_matches[i].queryIdx].pt);
    refPoint.push_back(keyPointRef[good_matches[i].trainIdx].pt);
}

Mat imgMatches;
drawMatches(currImg,keyPointCurr,ref,keyPointRef,good_matches,imgMatches,Scalar::all(-1),Scalar::all(-1),vector<char>(),DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS);
imshow("GoodMatches",imgMatches);

//Mat transformMatrix = estimateGlobalMotionRobust(curPoint,refPoint, 3);

Mat transformMatrix = estimateRigidTransform(curPoint,refPoint ,false); // false = rigid transform, no scaling/shearing

cout << transformMatrix << endl;
warpAffine(colorImg,outImg,transformMatrix,Size(frameW,frameH));
//warpPerspective(colorImg,outImg,transformMatrix,Size(frameW,frameH));

writeVideo.write(outImg);
imshow("Input",colorImg);
imshow("Stabilize",outImg);

if(waitKey(20) == 27)
{
    cout<<"ESC key is pressed by user" <<endl;
    break;
}
edit retag flag offensive close merge delete

Comments

I suggest taking a look at the master branch of openCV. It contains a comple video stabilization module. However documentation on it is poor so it won't be so straightforward to use.

StevenPuttemans gravatar imageStevenPuttemans ( 2014-02-12 13:30:32 -0500 )edit

What is the master branch of openCV? I read the opencv 3.0 document. It has videostab libary. http://docs.opencv.org/trunk/modules/videostab/doc/videostab.html But in fact I only get opencv 2.4.7 version. There are some inconsistent things with the videostabl module in version 3.0 document

If you have OpenCV 3.0 version, could you show it, please? I can not find it.

Jenny gravatar imageJenny ( 2014-02-12 23:05:06 -0500 )edit

Go to the github repository. There you can select branch, select master, which is basically a beta branch where stuff gets tested and changed before releasing it in stable clients. Guides on how to build openCV yourself can be found all over the forum and the internet.

StevenPuttemans gravatar imageStevenPuttemans ( 2014-02-13 01:32:07 -0500 )edit

I had the same problems using that technique. I switched to using the videostab example and it worked great.

Bryce gravatar imageBryce ( 2014-03-19 16:18:43 -0500 )edit

@Bryce: Could you share code, please? Thank you so much.

Jenny gravatar imageJenny ( 2014-03-20 03:22:59 -0500 )edit

The videostab example is in the repository under the c++ samples

StevenPuttemans gravatar imageStevenPuttemans ( 2014-03-20 05:33:15 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2014-04-23 15:21:10 -0500

In your "warpAffine(frame,warped,Transform_avg,Size( frame.cols, frame.rows));" function, you must specify FLAG as WARP_INVERSE_MAP for stabilization.

Sample code I have written:

Mat src, prev, curr, rigid_mat, dst;

VideoCapture cap("test_a3.avi");

while (1) { bool bSuccess = cap.read(src); if (!bSuccess) //if not success, break loop { cout << "Cannot read the frame from video file" << endl; break; }

cvtColor(src, curr, CV_BGR2GRAY);

if (prev.empty())
{
    prev = curr.clone();
}

rigid_mat = estimateRigidTransform(prev, curr, false);

warpAffine(src, dst, rigid_mat, src.size(), INTER_NEAREST|WARP_INVERSE_MAP, BORDER_CONSTANT);


// ---------------------------------------------------------------------------//

imshow("input", src);
imshow("output", dst);

Mat dst_gray;
cvtColor(dst, dst_gray, CV_BGR2GRAY);
prev = dst_gray.clone();

waitKey(30);

}

Hoping this will solve your problem :)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

Stats

Asked: 2014-02-12 12:27:51 -0500

Seen: 3,278 times

Last updated: Apr 23 '14