I am segmenting an image using GrabCut. However, Grabcut is very slow, so I am downscaling the image before running the algorithm and getting a mask. This mask is then upscaled and applied to the original image. I then put my segmented foreground into a background of purely transparent alpha value. This works decently well but the edges are super rough. What I mean is that, since my segmentation was done on a low res photo, the outline isn't perfect. What I'd like to do is transition the edges of my foreground so that the alpha value gradually goes from opaque foreground to transparent background.
Is there a good, established algorithm for transitioning alpha values from opaque foreground to transparent background? It seems almost like a blending algorithm without the background image.