If the sound of a bag rustling sends you into such a tailspin what does an overheard conversation do?
At some point you simply have to develop the power of concentration. Try focusing intently on your work rather than focusing intently on your surroundings.