Clean Extra added pixles on contour of text

edit

I had used the following code to do smoothing

//smooth the outer contour
    Mat smoothedMat;
    morphologyEx(_src,smoothedMat,MORPH_OPEN,getStructuringElement(MORPH_ELLIPSE,Size(3,3)));
    cv::bitwise_not(smoothedMat,smoothedMat);
    imwrite("c:\\temp\\smoothedMat.tif",smoothedMat);

and the result as the following the top right image is the original image the top left image is the smoothed image the bottom image is the combine image to see the diffirence between the two images

the smoothed image is better than the original , but not all pixels in the outer contour are remove , also some holes are filled also some regions (blobs) ar connected togther , is there any better way , to remove the outer noisy pixles without affacting the other pixles.