Ask Your Question

How to make transparent background While Stitching using homography

asked 2019-08-17 06:38:08 -0500

hede gravatar image

I have 2 images, I want to stitch them using homography. But I want there won't be any black border in the result image.

So I should use transparency. Do I need a create transparent background/mask? What I should do?

Example Stitched images:

image description

But I want to make transparent the black borders

This black borders: image description

How can I change my code?

My code:

cv::Mat stitch(const cv::Mat &img1,const cv::Mat &img2, cv::Mat &mask, const cv::Mat &H)
    //Coordinates of the 4 corners of the image
    std::vector<cv::Point2f> corners(4); 
    corners[0] = cv::Point2f(0, 0);
    corners[1] = cv::Point2f(0, img2.rows);
    corners[2] = cv::Point2f(img2.cols, 0);
    corners[3] = cv::Point2f(img2.cols, img2.rows);

    std::vector<cv::Point2f> cornersTransform(4);
    cv::perspectiveTransform(corners, cornersTransform, H);

    double offsetX = 0.0;
    double offsetY = 0.0;

    //Get max offset outside of the image
    for (size_t i = 0; i < 4; i++)
        std::cout << "cornersTransform[" << i << "]=" << cornersTransform[i] << std::endl;
        if (cornersTransform[i].x < offsetX)
            offsetX = cornersTransform[i].x;

        if (cornersTransform[i].y < offsetY)
            offsetY = cornersTransform[i].y;

    offsetX = -offsetX;
    offsetY = -offsetY;
    std::cout << "offsetX=" << offsetX << " ; offsetY=" << offsetY << std::endl;

    //Get max width and height for the new size of the panorama
    double maxX = std::max((double)img1.cols + offsetX, (double)std::max(cornersTransform[2].x, cornersTransform[3].x) + offsetX);
    double maxY = std::max((double)img1.rows + offsetY, (double)std::max(cornersTransform[1].y, cornersTransform[3].y) + offsetY);
    std::cout << "maxX=" << maxX << " ; maxY=" << maxY << std::endl;

    cv::Size size_warp(maxX, maxY);
    cv::Mat panorama(size_warp, CV_8UC3);

    //Create the transformation matrix to be able to have all the pixels
    cv::Mat H2 = cv::Mat::eye(3, 3, CV_64F);<double>(0, 2) = offsetX;<double>(1, 2) = offsetY;

    cv::warpPerspective(img2, panorama, H2*H, size_warp);

    //ROI for img1
    cv::Rect img1_rect(offsetX, offsetY, img1.cols, img1.rows);
    cv::Mat half;
    //First iteration
    if (mask.empty())
        //Copy img1 in the panorama using the ROI
        cv::Mat half = cv::Mat(panorama, img1_rect);

        //Create the new mask matrix for the panorama
        mask = cv::Mat::ones(img2.size(), CV_8U) * 255;
        cv::warpPerspective(mask, mask, H2*H, size_warp);
        cv::rectangle(mask, img1_rect, cv::Scalar(255), -1);
        //Create an image with the final size to paste img1
        cv::Mat maskTmp = cv::Mat::zeros(size_warp, img1.type());
        half = cv::Mat(maskTmp, img1_rect);

        //Copy img1 into panorama using a mask
        cv::Mat maskTmp2 = cv::Mat::zeros(size_warp, CV_8U);
        half = cv::Mat(maskTmp2, img1_rect);
        maskTmp.copyTo(panorama, maskTmp2);

        //Create a mask for the warped part
        maskTmp = cv::Mat::ones(img2.size(), CV_8U) * 255;
        cv::warpPerspective(maskTmp, maskTmp, H2*H, size_warp);

        maskTmp2 = cv::Mat::zeros(size_warp, CV_8U);
        half = cv::Mat(maskTmp2, img1_rect);
        //Copy the old mask in maskTmp2
        //Merge the old mask with the new one
        maskTmp += maskTmp2;

    return panorama;
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2019-08-20 07:02:37 -0500

What you have to do are these steps (@Witek his answer cuts off parts of the image, what you want to avoid)

  • Start by converting your image from BGR to BGRA using cvtColor which adds an alpha channels and which is read as transparancy in *.png images.
  • Find all pixels that have a fixed black color (which will almost never happen in natural images) and use those pixels to set their value in the alpha channel to 0.

The code below does this for white pixels, but can be easily switched.

// load as color image BGR
cv::Mat input = cv::imread("your image - can also be your image from the stitching result");

cv::Mat input_bgra;
cv::cvtColor(input, input_bgra, CV_BGR2BGRA);

// find all white pixel and set alpha value to zero:
for (int y = 0; y < input_bgra.rows; ++y)
for (int x = 0; x < input_bgra.cols; ++x)
    cv::Vec4b & pixel =<cv::Vec4b>(y, x);
    // if pixel is white
    if (pixel[0] == 255 && pixel[1] == 255 && pixel[2] == 255)
        // set alpha to zero:
        pixel[3] = 0;

// save as .png file (which supports alpha channels/transparency)
cv::imwrite("choose the location to store the image", input_bgra);
edit flag offensive delete link more

answered 2019-08-19 14:15:54 -0500

Witek gravatar image

I think you want to find the largest inscribed rectangle in a non-convex polygon (there are algorithms that can to that), but since you have the coordinates of your corner points you may use them in a simpler approach to define your rectangle. Please note that your rectangle will be based on two left corners of your left image and two right corners of your right image. The top-left vertical coordinate of your desired rectangle will be the largest of the vertical coordinates of your top corners. The top-left horizontal coordinate will be the largest of your horizontal coordinates of your left corners. If you follow this line of thinking you will find your rectangle.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2019-08-17 06:36:51 -0500

Seen: 47 times

Last updated: Aug 20