Would you put spaces around parentheses?
It would look ( to most people ) very strange.
It would look ( to most people ) very strange.
Em-dashes are not the same thing as parenthesis even if they are used in similar circumstances, and as a sibling comment recognized well, you'd still put spaces outside the parenthesis —but this would be terrible with em-dashes— wouldn't you agree?
This is really a style issue, and it mostly depends on what "school" you subscribe to.