This needn't be the case; it depends on the details and what the particular format is designed for.

Wasm is a good example of a bytecode format designed for small code size. Though I expect wasm may often still be larger than equivalent js just because it's so much lower level. But there's no reason you couldn't design a compact bytecode format with high level semantics, and I would expect it to beat any human readable format.