Ask Your Question
1

Image stitching of translating images

asked 2015-06-14 16:16:12 -0600

bjorn89 gravatar image

Hi all, I'm writing a program that stitch images taken by a flying drone. The problem is that images are translating like the drone is acting like a "scanner".So, when I calculate feature points and then homography, this one messes all my mosaic. There's a way in opencv (or with opencv together with another library) to stitch toghere images that differs by a translation instead a rotation?

edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted
0

answered 2015-06-15 10:38:33 -0600

LBerger gravatar image

updated 2015-06-17 00:36:15 -0600

with two images

int main(int argc, char** argv) {

Mat img1 = imread(mosaic_3.jpg", CV_LOAD_IMAGE_GRAYSCALE);
Mat img2 = imread("mosaic_2.jpg", CV_LOAD_IMAGE_GRAYSCALE);

std::vector<cv::KeyPoint> kOrb1;        /*<! Point clef de ORB */
std::vector<cv::KeyPoint> kOrb2;        /*<! Point clef de ORB */
cv::Mat descORB1;                   /*<! Descripteur associé à l'un des points clé ORB*/
cv::Mat descORB2;                   /*<! Descripteur associé à l'un des points clé ORB*/
vector<std::vector<cv::DMatch> > matches;   /*<! Descripteur appariés */

cv::Ptr<cv::Feature2D> b;
b = cv::ORB::create(2000);


b->detectAndCompute(img1, Mat(), kOrb1, descORB1);
b->detectAndCompute(img2, Mat(), kOrb2, descORB2);


cv::Ptr<cv::DescriptorMatcher> descriptorMatcher = cv::DescriptorMatcher::create("BruteForce");
matches.clear();
descriptorMatcher->knnMatch(descORB1, descORB2, matches, 1, Mat());
vector<Point> p, q;
Point2f u;
int nb = 0;
double seuil = 0;
for (int i = 0; i<matches.size(); i++)
    seuil += matches[i][0].distance;

seuil = seuil / matches.size()*0.5;
double d = 0;
for (int i = 0; i<matches.size(); i++)
{

    if (matches[i][0].distance<seuil)
    {
        u += kOrb1[matches[i][0].queryIdx].pt - kOrb2[matches[i][0].trainIdx].pt;
        p.push_back(kOrb1[matches[i][0].queryIdx].pt);
        q.push_back(kOrb2[matches[i][0].trainIdx].pt);
        nb++;
        if (nb>1)
            d += norm(p[nb - 1] - p[nb - 2]) / norm(q[nb - 1] - q[nb - 2]);
    }
}
d = d / (nb - 1);
Mat img;
resize(img2, img, Size(0, 0), 1, 1);
kOrb2.clear();
Mat descORB;
b->detectAndCompute(img, Mat(), kOrb2, descORB);
matches.clear();
descriptorMatcher->knnMatch(descORB1, descORB, matches, 1, Mat());
p.clear(); q.clear();
seuil = 0;
for (int i = 0; i<matches.size(); i++)
    seuil += matches[i][0].distance;
nb = 0;
seuil = seuil / matches.size()*0.5;
d = 0;
for (int i = 0; i<matches.size(); i++)
{

    if (matches[i][0].distance<seuil)
    {
        p.push_back(kOrb1[matches[i][0].queryIdx].pt);
        q.push_back(kOrb2[matches[i][0].trainIdx].pt);
        nb++;
        if (nb>1)
            d += norm(p[nb - 1] - p[nb - 2]) / norm(q[nb - 1] - q[nb - 2]);
    }
}
Mat_<double> v = estimateRigidTransform(p, q, false);// q=v*p; img2=v*img1
Mat_<double> h(3, 3);
h = Mat::eye(2, 3,CV_64F);
Mat id = Mat::eye(3, 3, CV_64F);
for (int i = 0; i < v.rows; i++)
for (int j = 0; i < v.cols; i++)
    h(j, i) = v(j, i);
cout << v << endl;
Mat res1((img1.rows + img2.rows), (img1.cols + img2.cols), img1.type());
Mat res2((img1.rows + img2.rows), (img1.cols + img2.cols), img1.type());
//    warpPerspective(img1, img1, id, Size(res.cols, res.rows), WARP_INVERSE_MAP);
warpAffine(img, res2, v, res2.size(), WARP_INVERSE_MAP);
warpAffine(img1, res1, h,res1.size(), WARP_INVERSE_MAP);
imshow("Res", max(res1,res2));
imshow("img1", img1);
imshow("img2", img2);
waitKey();
return 0;

}

edit flag offensive delete link more

Comments

Hi! It does not work,I have only a part of image and the other half is black. I see that I need rotation too. I can't use homography because it will ruin my stitch/mosaic with strange warping.. how can I add rotation?

bjorn89 gravatar imagebjorn89 ( 2015-06-15 14:27:29 -0600 )edit
1

Yes translation is not enough. Finally I think the shortest way it's to use your sensor on your drone like say @StevenPuttemans. you can fit transaltion with above program and evaluate rotation using inclinometer and magnetometer.

LBerger gravatar imageLBerger ( 2015-06-16 08:52:42 -0600 )edit

I've found that estimateRigidTransform gives a matrix with rotation and translation.it works pretty well, but here's the problem: I need to orthorectify the images. How can it be done?

bjorn89 gravatar imagebjorn89 ( 2015-06-16 10:54:22 -0600 )edit
1
LBerger gravatar imageLBerger ( 2015-06-17 00:45:10 -0600 )edit
1

@LBerger I've seen the web page you linked above, and that's what exactly happens to me! I've found orfeo before but I didn't knew it was the solution. I give a try and let you know. Thanks a lot man!

bjorn89 gravatar imagebjorn89 ( 2015-06-17 02:33:43 -0600 )edit

Reading the documentantions seems that I need DEM informations,which I don't have. What a mess XD

bjorn89 gravatar imagebjorn89 ( 2015-06-17 05:34:37 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2015-06-14 16:16:12 -0600

Seen: 1,602 times

Last updated: Jun 17 '15