My university made us use really crappy power supplies and dev boards. Nothing worked unless you first put a large bulk capacitor on the power supply's output, and small capacitors close to the components.

Also I got bitten by parasitics in capacitors very early in my career: capacitors of different face value will resonate with each other to effectively kill the decoupling network at a specific frequency (resulting, for me, in an amplifier with a nice hole in its frequency response).