Sure, it might work. A binary format still has to be parsed, of course, but it would likely be a little more efficient.

One question is whether that efficiency gain would be worth the hassle of a binary format. Binary formats have generally not been a good fit for the web; DNS and HTTP/2 are the only widespread protocols I can remember off the top of my head that are binary. gRPC is binary, and look at the lack of mature toolchains to work with it -- while there are some tools now, the selection is minuscule compared to those available for HTML and CSS.

Binary formats are also trickier to extend and keep backwards-compatible.