ORB feature detection/Image comparison : Should I scale images?

edit

So I'm using ORB keypoints and feature descriptors in a video comparison algorithm. It works pretty well. However, I wonder if it is necessary to scale each key-frame to 1920x1080 before I feed it into the keypoint and feature descriptor methods. Sometimes tells me that accounts for a considerable chunk of the signature cration time. Am I helping the algorithm or wasting time and hurting the accuracy be scaling all frames to be the same resolution?