I've reverse engineered some protocols -- haven't done a USB one yet, but I'm sure the principals are similar. Grab some data, look at it (usually with a good hex editor -- last time I used ghex2), look for patterns. Usually there's some sort of packet structure, or maybe some data that looks like something in ascii -- ghex2 shows you what every 2 bytes or 4 bytes are if they were signed/unsigned ints, floats etc. Usually something will jump out at you.

It's a fun game usually.