Alpha-Dependent Template Matching

asked 2013-07-10 22:53:33 -0500

awu gravatar image

updated 2013-07-10 22:54:36 -0500

I have a couple of pics that are either 8UC3 or 8UC4 and I would like to template match them almost exactly. I have done this in my code, and I would like to learn if there are better methods to optimize and simplify the code? In also know that it is currently done with block methods and fft, and also I have read about phase based matching. (CUDA's next, but this will do for now). I think I have moved everything out of the loop that is possible. Breaking during GDB sessions usually end up on the multiply or the merge in the loop. I also haven't guarded against all possible values of different types and perhaps there are better ways to guard against different types? (I think my code will fail for graychannel with alpha). Last, what is the best way to guard against matching against a black patch since everything will be 0.

I have also referred to these codes, but I developed my own:


alpha-dependent template matching

opencv template matching and transparency

Mat templ = _templ.getMat();
Mat img = _img.getMat();
const double UCHARMAX = 255;
const double UCHARMAXINV = 1./UCHARMAX;
Mat templRed(templ.size(), CV_8UC1);
Mat templGreen(templ.size(), CV_8UC1);
Mat templBlue(templ.size(), CV_8UC1);
Mat templAlpha(templ.size(), CV_8UC1);

Mat imgRed(img.size(), CV_8UC1);
Mat imgGreen(img.size(), CV_8UC1);
Mat imgBlue(img.size(), CV_8UC1);
Mat imgAlpha(img.size(), CV_8UC1);
if(templ.depth() == CV_8U && img.depth() == CV_8U && 
  (img.type() == CV_8UC3 || img.type() == CV_8UC4) &&
  (templ.type() == CV_8UC3 || templ.type() == CV_8UC4)){
  if(templ.type() == CV_8UC3){
    templAlpha = Scalar(UCHARMAX);
    cvSplit(&templ, &templRed, &templGreen, &templBlue, NULL);
  else if(templ.type() == CV_8UC4){
    cvSplit(&templ, &templBlue, &templGreen, &templRed, &templAlpha);
  if(img.type() == CV_8UC3){
    imgAlpha = Scalar(UCHARMAX);
    cvSplit(&img, &imgRed, &imgGreen, &imgBlue, NULL);
  else if(templ.type() == CV_8UC4){
    cvSplit(&img, &imgBlue, &imgGreen, &imgRed, &imgAlpha);
  Size corrSize(img.cols - templ.cols + 1, img.rows - templ.rows + 1);
  _result.create(corrSize, CV_32F);
  Mat result = _result.getMat();

  Mat tempRed=templRed.clone();
  Mat tempBlue=templBlue.clone();
  Mat tempGreen=templGreen.clone();

  multiply(tempRed, templAlpha, tempRed, UCHARMAXINV);
  multiply(tempGreen, templAlpha, tempGreen, UCHARMAXINV);
  multiply(tempBlue, templAlpha, tempBlue, UCHARMAXINV);

  multiply(imgRed, imgAlpha, imgRed, UCHARMAXINV);
  multiply(imgGreen, imgAlpha, imgGreen, UCHARMAXINV);
  multiply(imgBlue, imgAlpha, imgBlue, UCHARMAXINV);

  Mat temp(templ.size(), CV_8UC3);
  cvMerge(&tempBlue, &tempGreen, &tempRed, NULL, &temp);
  const double tempsum2 = norm(temp);

  Mat ROI(templ.size(), CV_8UC3);

  for(int i=0; i<img.rows-templ.rows; i++){
    for(int j=0; j<img.cols-templ.cols; j++){

      const Rect rect = Rect(j, i, templ.cols, templ.rows);
      Mat ROItempRed(imgRed, rect);
      Mat ROItempGreen(imgGreen, rect);
      Mat ROItempBlue(imgBlue, rect);

      Mat ROIRed = ROItempRed.clone();
      Mat ROIBlue = ROItempBlue.clone();
      Mat ROIGreen = ROItempGreen.clone();

      multiply(ROIRed, templAlpha, ROIRed, UCHARMAXINV);
      multiply(ROIGreen, templAlpha, ROIGreen, UCHARMAXINV);
      multiply(ROIBlue, templAlpha, ROIBlue, UCHARMAXINV);

      cvMerge(&ROIBlue, &ROIGreen, &ROIRed, NULL, &ROI);
      Mat corr(result, Rect(j, i, 1, 1));
      crossCorr( ROI, temp, corr, corr.size(), corr.type(), Point(0,0), 0, 0);
      const double normalization = norm(ROI)*tempsum2;
      if(normalization != 0)
        corr /= normalization;
        corr = 0.; //How should black be matched?
//...  else ...
edit retag flag offensive close merge delete


Would you mind also posting the crossCorr function that you used? I attempted to replace it and the for loops by MatchTemplate, but the result is innacurate. Thanks!

LadyZayin gravatar imageLadyZayin ( 2013-07-26 13:34:44 -0500 )edit

Nevermind, I got it to work with MatchTemplate. I was experiencing problems because the images I am matching are black and white, and I wanted to mask the white regions. As a result, the alpha channel would generate a completely black image. I just had to invert my images before applying the mask.

LadyZayin gravatar imageLadyZayin ( 2013-07-29 12:25:10 -0500 )edit

@LadyZayin Apologize for the late response, I didn't notice any notification on the comment. The crossCorr function I used is the same one in OpenCV, but glad that you solved the problem.

awu gravatar imageawu ( 2013-08-29 18:55:58 -0500 )edit

No problem, thanks anyway! :)

LadyZayin gravatar imageLadyZayin ( 2013-11-18 22:53:59 -0500 )edit

@awu@LadyZayin Hi awu, LadyZayin, can you send a pull request to OpenCV Github repo to add these source codes to OpenCV? Alexander has raised a feature request here: Thanks!

patrickkw gravatar imagepatrickkw ( 2014-09-19 22:32:20 -0500 )edit

please change cvSplit and cvMerge to cv::split and cv::merge, since the old c-api functions won't survive the port to opencv3.0

berak gravatar imageberak ( 2014-09-20 02:53:46 -0500 )edit