It's amazing that it can take you 3 tries to plug in a USB cable which only has 2 correct ways it can be oriented.
The plastic bit inside of the plug usually goes on the side facing whatever board it's interfacing with.
For example, looking straight on from the back of an ATX case, it's usually on the left side - where the motherboard mounts
Front panel connectors are anarchy... but at least they're usually visible
Sideways ports may be a crap shoot I'm not sure if their orientation is standardized. And I'm not sure about modern USB version if that's still the case of logo faces up.