1) In any wireless device, it is not hard to imagine a bug causing problems
2) In any electronic device carrying oscillating currents, there is EM emission, therefore it is possible for unintended leakage on bad frequencies (e.g. this trace is just the right length to be a broadcast antenna at frequency Y, given normal modes of operation.
3) Some devices may individually not cause problems, even though they are low power emissions on certain bands, but cumulatively they may cause signal degradation in those bands.
4) Having and enforcing them forces engineers to ensure they are met up front, resulting in better, nicer playing devices to begin with, as it is cheaper to engineer it right, than to deal with the consequences of doing it poorly.
It turned out I wasn't going mad, the thing was picking up local CB radio.