Fundamentally I wouldn't say octal is fundamentally easier to interpret other than hex because it's just half as many digits. The shorthand things about octal used in unix you mentioned all still technically apply to hex as well. The only really tricky part is where it becomes letters and you have to think if D or E is the odd value with the first bit set. If you were doing hex all day though, the 6 hex letter patterns would be pretty easy to memorize.
Personally I would say for front panel systems specifically was just about physical space for the switches. If you look up CPU trainers for something like Intel 8080 in the Altair 8800, you'll see it uses a hex keypad. Multiple 16b values would have been a lot harder to fit in single bit switches.