You'd want to make it so either:

a) There is nothing to scroll in the first place, so you don't need a scrollbar

b) Capture the pointer so all interaction only occurs within the game's area.