Power supplies handle that badly, and the pulses turn your supply traces in a big antenna and suddenly you're an unlicensed FM radio station.
So for the capacitor that's put in place to buffer that load, the total capacity is important, but what really matters is that the part can manage that 100MHz charge/discharge cycle.
Another cool insight is around "dual" relationships in circuits. You can often think of some components as roughly analogous to each other, with a switch of current and voltage. For example, if you connect a capacitor in parallel with a power supply and load, the voltage across it will increase as a function of time until it reaches the power supply voltage. Similarly, if you connect an inductor in series with a power supply and load, the current through it will increase as a function of time until it reaches the current that would otherwise pass through the load.
Like most good analogies, it's not 100% airtight, but it provides useful insights.
At least, that's my understanding of it.