Tackle low FPS for correlation code to compute shift in Image

edit

I am trying to track an object using correlation. I am finding a smaller patch in a larger image, frame by frame. For this, I am finding the shift in the patch, and where correlation is maximum, update the patch with a new patch.

My code is:

cv::Mat im_float_2,imagePart_out;
cv::Mat im_floatBig;
cv::Scalar im1_Mean, im1_Std, im2_Mean, im2_Std;

double covar, correl;
int n_pixels;

void computeShift()
{
    int maxRow=0, maxCol=0, TX, TY;
    double GMAX=0;
    Mat image_window = Mat::zeros(imagePart.rows, imagePart.cols, CV_32F);

    imagePart.convertTo(im_float_2, CV_32F);
    imageBig.convertTo(im_floatBig,CV_32F);

    for(maxRow=0; maxRow<=imageBig.rows-image_window.rows;maxRow++)
    {
        for(maxCol=0; maxCol<imageBig.cols-image_window.cols;maxCol++)
        {

            image_window = im_floatBig( cv::Rect( maxCol, maxRow, 
            image_window.cols, image_window.rows ) );

            n_pixels = image_window.rows * image_window.cols;

            // Compute mean and standard deviation of both images

            meanStdDev(image_window, im1_Mean, im1_Std);
            meanStdDev(im_float_2, im2_Mean, im2_Std);

            // Compute covariance and correlation coefficient
            covar = (image_window - im1_Mean).dot(im_float_2 - im2_Mean) / n_pixels;

            correl = covar / (im1_Std[0] * im2_Std[0]);
            if (correl > GMAX)
            {
            GMAX = correl; TX = maxRow; TY=maxCol;
            image_window.convertTo(imagePart, CV_8UC1);
            }
        }
    }

            cvtColor(imagePart, imagePart_out, CV_GRAY2BGR);
            printf("\nComputed shift: [%d, %d] MAX: %f\n", TX, TY,GMAX);                

}

But when executing this I am getting very low FPS(1-2) even for small video size (Frame size-262x240, Patch size- 25x25).

Is there any way to achieve higher FPS. I am also looking in the direction of phase correlation, but not sure how to go about it from here. Can converting it to frequency domain will help?

For now, I want to optimize the above code for speed.