Ask Your Question

How to compute the translation and rotation in stitching(opencv module)?

asked 2016-11-24 21:23:54 -0500

AlexChan gravatar image

In the opencv module (stitch_detail.cpp):

warper->warp(img, K, cameras[img_idx].R, INTER_LINEAR, BORDER_REFLECT, img_warped);

Img_warped is the output image, img is the input image. With the cameras[img_idx].R , we can compute the rotation of each axis :theta_x, theta_y, theta_z. But how to compute the translation of each axis? Does anybody has some tips?

As far as I know, the cameras[img_idx].t does not contain the translation information. I had modified the t, but nothing happen. (stitch_detail.cpp): if (blender.empty()) { blender = Blender::createDefault(0, try_gpu); Size dst_sz = resultRoi(corners, sizes).size(); float blend_width = sqrt(static_cast<float>(dst_sz.area())) * 5 / 100.f; if (blend_width < 1.f) blender = Blender::createDefault(Blender::NO, try_gpu); blender->prepare(corners, sizes); } // Blend the current image blender->feed(img_warped_s, mask_warped, corners[img_idx]);

I find corners and sizes can influence the result. The corners is a vector<Point>, it is a set of the left-and-top points of sub-images. The sizes is a vector<Size>, it is a set of the size of each sub-image. They are modified in warper->warpRoi (stitch_detail.cpp)

// Update corner and size
                Size sz = full_img_sizes[i];
                if (std::abs(compose_scale - 1) > 1e-1)
                    sz.width = cvRound(full_img_sizes[i].width * compose_scale);
                    sz.height = cvRound(full_img_sizes[i].height * compose_scale);
                Mat K;
                cameras[i].K().convertTo(K, CV_32F);
                **Rect roi = warper->warpRoi(sz, K, cameras[i].R); //** 
                corners[i] =;
                sizes[i] = roi.size();

The code of warper->warpRoi:( warper_inl.hpp )

template <class P>
Rect RotationWarperBase<P>::warpRoi(Size src_size, const Mat &K, const Mat &R)
    projector_.**setCameraParams**(K, R);
    Point dst_tl, dst_br;
    **detectResultRoi**(src_size, dst_tl, dst_br);
    return Rect(dst_tl, Point(dst_br.x + 1, dst_br.y + 1));

The code of projector_.setCameraParams(K, R):( warper_inl.hpp )

void ProjectorBase::setCameraParams(const Mat &K, const Mat &R, const Mat &T)
    CV_Assert(K.size() == Size(3, 3) && K.type() == CV_32F);
    CV_Assert(R.size() == Size(3, 3) && R.type() == CV_32F);
    CV_Assert((T.size() == Size(1, 3) || T.size() == Size(3, 1)) && T.type() == CV_32F);

    Mat_<float> K_(K);
    k[0] = K_(0,0); k[1] = K_(0,1); k[2] = K_(0,2);
    k[3] = K_(1,0); k[4] = K_(1,1); k[5] = K_(1,2);
    k[6] = K_(2,0); k[7] = K_(2,1); k[8] = K_(2,2);

    Mat_<float> Rinv = R.t();
    rinv[0] = Rinv(0,0); rinv[1] = Rinv(0,1); rinv[2] = Rinv(0,2);
    rinv[3] = Rinv(1,0); rinv[4] = Rinv(1,1); rinv[5] = Rinv(1,2);
    rinv[6] = Rinv(2,0); rinv[7] = Rinv(2,1); rinv[8] = Rinv(2,2);

    Mat_<float> R_Kinv = R * K.inv();
    r_kinv[0] = R_Kinv(0,0); r_kinv[1] = R_Kinv(0,1); r_kinv[2] = R_Kinv(0,2);
    r_kinv[3] = R_Kinv(1,0); r_kinv[4] = R_Kinv(1,1); r_kinv[5] = R_Kinv(1,2);
    r_kinv[6] = R_Kinv(2,0); r_kinv[7] = R_Kinv(2,1); r_kinv[8] = R_Kinv(2,2);

    Mat_<float> K_Rinv = K * Rinv;
    k_rinv[0] = K_Rinv(0,0); k_rinv[1] = K_Rinv(0,1); k_rinv[2] = K_Rinv(0,2);
    k_rinv[3] = K_Rinv ...
edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted

answered 2016-11-25 01:29:00 -0500

LBerger gravatar image

there is no translation in stitching module because there is no translation in this paper :

Matthew Brown and David G Lowe. Automatic panoramic image stitching using invariant features. International journal of computer vision, 74(1):59–73, 2007. and page 2 it is written :

Assuming that the camera rotates about its optical centre, the group of transformations the images may undergo is a special group of homographies.

If you want to include transaltion in your code you should forget this code.

You can start by making a panoramic view with a stereovision system

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2016-11-24 21:23:54 -0500

Seen: 559 times

Last updated: Nov 25 '16