... build a smart boot device that provides a bunch of different CD images for BIOSes that don't do USB boot well
... build a safe password keeper that looks like a HID device
... do something interesting with USB-based gamepads to map those to keyboard keys for playing old DOS games
... make a connected USB storage device that offers some sort of additional sync services for the data stored on it (maybe over wifi?)
... make a network adapter that also acts as a firewall, or possibly links to tor/vpn/etc
... Something something MIDI over USB