This is why I only listen to the instrumental music during work, and pain to hear an occasional phrase dropped into a perfectly good IDM / synthwave / electronica track. I wish a "speech-free" tag existed e.g. on Bandcamp; some artists helpfully put annotations on their albums, e.g. Master Boot Record's "100% dehumanized", but apparently most artists and likely listeners do not see the presence of words as a large differentiator.
I know 2-3 of my friends who exhibit a similar effect. I wonder if it's rare, or not so rare but not noticed much.