Revision history [back]

Using transform matrix from larger area on ROI

I've a problem that I don't really know how to solve. The situation is that I've got a picture, inside this picture I have one ROI (a large rect). Inside this rect I then have X number of smaller ROI:s.

I then use the getPerspectiveTransform() method on the larger of the ROI. What I then want to do is to apply this matrix on the smaller ROI:s in-order to warp them separately but using the matrix from the larger ROI. The reason I want to do this is basically because I want to keep the bounding rects I've got before the warp. If there is any way to warp the larger ROI and keep the bounding rects inside of this (i.e. keep the smaller ROI:s) that would be very helpful!

Anyway, this is what I've tried and the results it produces:

vector<Mat> warpSmallerRoi( vector<Rect> smallerRoi ) { 

    vector<Mat> quad;

    for( int i = 0; i < smallerRoi.size(); ++ i ) {
        Mat wholeQuad;

        wholeQuad = Mat::zeros( originalImage(smallerRoi[i]).rows, originalImage(smallerRoi[i]).cols, CV_8UC1 );
        vector<Point2f> largerRoiCorners;
        vector<Point2f> quadPoints;

        // Takes the corners from the first smaller ROI and the last smaller ROI, represents the bigger ROI
        largerRoiCorners.push_back( Point2f( smallerRoi[0].tl().x, smallerRoi[0].tl().y ) ); // Top left
        largerRoiCorners.push_back( Point2f( smallerRoi[ smallerRoi.size() - 1 ].br().x, smallerRoi[ smallerRoi.size() - 1  ].tl().y ) ); // Top right
        largerRoiCorners.push_back( Point2f( smallerRoi[ smallerRoi.size() - 1 ].br().x, smallerRoi[ smallerRoi.size() - 1  ].br().y ) ); // Bottom right
        largerRoiCorners.push_back( Point2f( smallerRoi[0].tl().x, smallerRoi[0].br().y ) ); // Bottom left

        quadPoints.push_back( Point2f(0, 0) );
        quadPoints.push_back( Point2f(wholeQuad.cols, 0) );
        quadPoints.push_back( Point2f( wholeQuad.cols, wholeQuad.rows ) );
        quadPoints.push_back( Point2f(0, wholeQuad.rows) );

        // Transform matrix for the larger ROI, this warps into a perfect result.
        Mat largerRoiTransformMatrix = getPerspectiveTransform( largerRoiCorners, quadPoints );

        //Corners of the smaller ROI
        vector<Point2f> corners;
        corners.push_back( Point2f( smallerRoi[i].tl().x, smallerRoi[i].tl().y ) ); // Top left
        corners.push_back( Point2f( smallerRoi[i].br().x, smallerRoi[i].tl().y ) ); // Top right
        corners.push_back( Point2f( smallerRoi[i].br().x, smallerRoi[i].br().y )); // Bottom right
        corners.push_back( Point2f( smallerRoi[i].tl().x, smallerRoi[i].br().y ) ); // Bottom left

        Mat transformMatrix = getPerspectiveTransform( corners, quadPoints );

        /* This part is just experimental and does not work all the time, it works well sometimes though 
           Uses the pars from the larger ROI transform matrix and applies it on the smaller ROI
           then warps it.
        */
        transformMatrix.at<double>(1,0) = largerRoiTransformMatrix.at<double>(1,0);
        transformMatrix.at<double>(1,1) = largerRoiTransformMatrix.at<double>(1,1);
        transformMatrix.at<double>(1,2) = largerRoiTransformMatrix.at<double>(1,2);

        transformMatrix.at<double>(2,0) = largerRoiTransformMatrix.at<double>(2,0);
        transformMatrix.at<double>(2,1) = largerRoiTransformMatrix.at<double>(2,1);
        transformMatrix.at<double>(2,2) = largerRoiTransformMatrix.at<double>(2,2);

        // Warps the 
        warpPerspective( plateRgb, wholeQuad, transformMatrix, wholeQuad.size() );
        quad.push_back(wholeQuad);
    }
    return quad;
}

This function works sometimes, especially when the larger ROI is already pretty straight (I guess this is because my replacements are not that big difference from the original values then. Eg:

From:

Smaller ROI without warp

To:

Smaller ROI with warp

But then when the larger ROI has much skew the result is not so good:

From:

Smaller ROI, not good, without warp

To:

Smaller ROI, not good, with warp

As you can see the right part of the "H" is here a bit outside of the image. How should I go forward to transform my ROI:s so that the "H" (and all others) fit into the image and are warped with the correct tranformation matrix?

Sorry if I missed out on any information, ask in that case! Thanks :)