What options exist for removing Crowds/Audiences from Videos

edit

Could the community suggest methods for removing or reducing the influence crowds or audiences from videos as a preprocessing step? If you take this image as an discussion example and assume its a video of a game.

1. what options are available in OpenCV to remove the crowd in the background without manually cropping the frame
2. Which of those options could be used for multi angle environments (assume there was a camera in an elevated position in corner of the stand behind the white team and looking down at the game) so the crowd is now appearing on a diagonal angle to the playing area with the playing area taking up most of the frame
3. What options could be carried out without manual intervention

Could you flip the problem and remove everything thats not within X pixels of the dominate playing area colour and what would the OpenCV code for that look like?

The image came from here