remap() gives pixelated result

edit

I'm applying a warp using OpenCV's remap() function, but the result looks quite blocky. I'm using bicubic sampling using a float matrix to shift the x & y like this:

remap(src, dst, mapx, mapy, INTER_CUBIC, BORDER_CONSTANT);

As an example:

(original, dst & zoomed in result)

The scaling factor is quite high on this example, and if you look at the bottom of the shirt it fails to do the bicubic sampling effectively. Is this because the scaling factor is high, and so the pixels sampled for bicubic interpolation are too few, hence resulting in this jagged result?

How can I make this nice & smooth?