Man up and talk to him about it if it's interrupting your productivity. I'm sure he cares about your productivity.