I've got an early generation portable USB drive that I've kept for some nostalgia.
https://imgur.com/2XcXCPe - its 128 MB.
I purchased in '96 (or maybe '97). You'll note that it is orange which indicated solid state. There was also a black one which had a spinning disk in that same form factor and held 256 MB.
I used it a few times in between 2010 and 2015 for transferring a file around sneaker net. You'll note that the USB plug is on a short cord - the places where I had to plug it in didn't always allow for easily inserting a short rectangular stick - they either stuck out too far for where the computer was or required moving other things out of the way. This one let me plug it in with minimal obstruction to other things in a cluttered USB port area.