Another important consideration are field names. If you use string field names, these take up more space and are more expensive to access (dictionary with string keys). Though compression and precomputed perfect hashing based dictionaries can mitigate those downsides surprisingly well. Simple integer keys like in protobuf are a bit cheaper to look up and more compact as well. And offset based approaches like capnproto can reduce those costs even further, though they come with their own limitations.
In general there are many good-enough solutions, but if you want the ideal solution you have to think about many details (zero-copy, single-pass write, random access read, human readability, extensibility etc.) with contradictory requirements on the format.