How can I rewrite this warp-affine using OpenCV?

asked 2017-04-12 03:52:57 -0500

lovaj gravatar image

updated 2017-04-12 04:29:13 -0500

I'm trying to optimize this code, in particular:

bool interpolate(const Mat &im, float ofsx, float ofsy, float a11, float a12, float a21, float a22, Mat &res)
{         
   bool ret = false;
   // input size (-1 for the safe bilinear interpolation)
   const int width = im.cols-1;
   const int height = im.rows-1;
   // output size
   const int halfWidth  = res.cols >> 1;
   const int halfHeight = res.rows >> 1;
   float *out = res.ptr<float>(0);
   for (int j=-halfHeight; j<=halfHeight; ++j)
   {
      const float rx = ofsx + j * a12;
      const float ry = ofsy + j * a22;
      for(int i=-halfWidth; i<=halfWidth; ++i)
      {
         float wx = rx + i * a11;
         float wy = ry + i * a21;
         const int x = (int) floor(wx);
         const int y = (int) floor(wy);
         if (x >= 0 && y >= 0 && x < width && y < height)
         {
            // compute weights
            wx -= x; wy -= y;
            // bilinear interpolation
            *out++ = 
               (1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x)   + wx * im.at<float>(y,x+1)) +
               (       wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));
         } else {
            *out++ = 0;
            ret =  true; // touching boundary of the input            
         }
      }
   }
   return ret;
}

According to Intel Advisor, this is a very time consuming function. In this question I asked how I could optimize this, and someone made me notice that this is warp-affine transformation.

Now, since I'm not the image processing guy, I had to read this article to understand what a warp-affine transformation is.

To my understanding, given a point p=(x,y), you apply a transformation A (a 2x2 matrix) and then translate it by a vector b. So the obtained point after the transformation p' can be expressed as p' = A*p+b. So far so good.

However, I'm a little bit confused on how to apply cv::warpAffine() to this case. First of all, from the function above interpolate() I can see only the 4 A components (a11, a12, a21, a22), while I can't see the 2 b components...Are they ofsx and ofy?

In addition notice that this function returns a bool value, which is not returned by warpAffine (this boolean value is used here at line 126), so I don't know I could this with the OpenCV function.

But most of all I'm so confused by for (int j=-halfHeight; j<=halfHeight; ++j) and for(int i=-halfWidth; i<=halfWidth; ++i) and all the crap that happens inside.

I understand that:

        // bilinear interpolation
        *out++ = 
           (1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x)   + wx * im.at<float>(y,x+1)) +
           (       wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));

Is what INTER_LINEAR does, but apart from that I'm totally lost.

So, to test my approach, I tried to do the equivalent of line 131 of this as:

     bool touchesBoundary = interpolate(smoothed, (float)(patchImageSize>>1), (float)(patchImageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);
     Mat warp_mat( 2, 3, CV_32FC1 );
     warp_mat.at<float>(0,0) = imageToPatchScale;
     warp_mat.at<float>(0 ...
(more)
edit retag flag offensive close merge delete

Comments

btw, buffer overflow here:

if (x >= 0 && y >= 0 && x < width && y < height)

needs to be:

if (x >= 0 && y >= 0 && x < width-1 && y < height-1) // since it uses x+1,y+1 below</strike>
berak gravatar imageberak ( 2017-04-12 09:47:40 -0500 )edit
1

@berak thanks for your comment, but this: // input size (-1 for the safe bilinear interpolation) const int width = im.cols-1; const int height = im.rows-1;

at the begin shouldn't solve it?

lovaj gravatar imagelovaj ( 2017-04-12 12:44:00 -0500 )edit

@berak btw here someone suggested that this can be replaced with a simple cv::warpAffine() but I'm a little bit confused when it's said that the origin is the center of the image and how to solve this...can you help me?

lovaj gravatar imagelovaj ( 2017-04-12 12:57:47 -0500 )edit

ah right, missed that. ignore comment above.

berak gravatar imageberak ( 2017-04-12 13:01:12 -0500 )edit