Today, the best audio drivers are external USB DACs and amplifiers. You can extend your storage with GB of USB flash storage. USB->Serial has kept up as a legacy option (although its latency issues and Win7 transition has made things difficult to work with in practice... gotta keep that old WinXP box to work with a lot of Serial stuff. But that's not USB's fault per se and more about Windows Vista's new security model).
Audio (DAC, Headphones, Synthesizers, USB->MIDI, Instruments), Storage (USB Flash Drives, USB Hard Drives), video game peripherals (XBox Controllers, PS4 Controllers, Joysticks, Driving Wheels), even legacy (USB->Serial Modems if you really need it) are all supported from the 5Gbps USB3.0 port.
--------
PCMCIA was needed in the 90s because computing power wasn't good enough. DMA transfers, directly to the computer's RAM, was the only way to get latency / bandwidth requirements of practical devices.
Furthermore, hardware interfaces were obscure. Your "Jazz drive" external storage was SCSI based and required a specific device. USB internet modems were years away from being invented (and thus you used PCMCIA modems on those laptops).
In essence... PCMCIA was the way you attached external devices to laptops back then. More so than USB or Firewire. Modern interfaces are more secure, more inter-operable, and faster.
I mean, those flash-sticks require a rather beefy CPU to handle all of the USB-packets that go back and forth between the OS and the USB stick. That level of miniature processor power just didn't exist in the 90s, so "dumb" hardware attachments had to use simple direct-to-memory transfers through a relatively simple bus. I wouldn't be surprised if a modern USB-stick's processor had more computing power than 90s-era laptops.