Template matching is not working correctly?Is there a way to solve this

edit

Hi,

I am using opencv for template matching.Actually i want to match the left hand or right hand of the person exactly by comparing it with sample left hand or right hand template that i have took and seperate his hand alone from that original image.

The following function is used for template matching:-

public void Template_Matching(Image template)

{

using (Image result = openCVImg.MatchTemplate(template, Emgu.CV.CvEnum.TM_TYPE.CV_TM_CCOEFF))

    {

        double[] minValues, maxValues;

        System.Drawing.Point[] minLocations, maxLocations;

        result.MinMax(out minValues, out maxValues, out minLocations, out maxLocations);

        var bytes = result.Bytes;

        if (maxValues[0] > 1.1)
        {
            istracking = true;

            // This is a match. Do something with it, for example draw a rectangle around it.
         //   System.Drawing.Rectangle match = new System.Drawing.Rectangle(maxLocations[0].X, maxLocations[0].Y, template.Width / 2, template.Height);
            System.Drawing.Rectangle match = new System.Drawing.Rectangle(new System.Drawing.Point(maxLocations[0].X, maxLocations[0].Y), new System.Drawing.Size(template.Width, template.Height));

            openCVImg.Draw(match, new Bgr(System.Drawing.Color.Red), 1);
            //   imageToShow.DrawPolyline(minLocations, false, new Bgr(System.Drawing.Color.Red), 3);

        }
        else
        {
            istracking = false;
        }

    }

}

But the above function is not working correctly.For example in the below picture,

Fig 1:- i have compared the hand template with the person showing hand.Now it is matching correctly and the rectangle is drawn over the hand of the person.

Fig 2:- I have compared the hand template with closed hand of that person.But here it is highlighting the head of the person.(Why?)

Consider the person is in sleeping mode, My goal is to remove the left hand and right hand of the person simultaneously by matching the left hand of right hand template of any other person.I want to do pixel by pixel comparison of these two pictures and remove the left hand and right hand completely from the original image.

Can you please suggest me any other approach to acheive my goal?

handtrack.png handtrackwrong.png