ASCII protocol buffers as config files | Hacker News Reader

ASCII protocol buffers as config files

rachelbythebay.com

44 pointszdw17 comments

There's no official spec for the text format, except for the one reverse engineered from 3,000 lines of C++:

https://protobuf.dev/reference/protobuf/textformat-spec/

Looks like HCL in the end, mostly.

If HCL was more standardised across languages, maybe without the interpolation bits, it would be a nice common configuration format IMHO [0].

[0]: https://www.linode.com/docs/guides/introduction-to-hcl/

There’s a solid implementation for Rust: https://docs.rs/hcl-rs/latest/hcl/

And an implementation for python which is at least widely used, not sure on quality: https://pypi.org/project/python-hcl2/

We do this at my work, it works fine. Much better than JSON and skips the insanity of YAML. And it’s nice to have a commented definition file for what is expected/valid in the config.

I think JSON5 is a better choice for this than this weird protobuf thing.

It's also a one-liner to convert JSON5 to normal json or yaml for things that want it that way.

I’m not clear why this recommends using required on everything - I though protobuf v3 had removed that and made everything default to 0 (or equivalent) if not present on the wire.

I'm starting to suspect the state of Computer Science is devolving.

I might be an outlier here, but except when data size is a real constraint, I tend to favor XML. (ToML being a good contender, but I don't have enough real-life experience with it yet)

    [Drake turning away
     with his hand up]  Protobuf as config file format

    [Drake smiling]     SQLite3 as config file format

Anyone who actually knows protobuf knows this is a comically bad idea. Protobuf is not self describing, and even the types are ambiguous. And pointlessly ambiguous. The spec itself admits that they have two extra bits in the wire format, either one could be used to identify embedded messages.

As someone who wrote a protobuf parser, Please don't use protobuf for anything, ever.

I'm not convinced. The ASCII representation looks human readable to me, and the associated .proto definitions will make it easy to parse in any language you want. Wire format doesn't sound relevant to the plaintext representation (or am I wrong?).

I've never tried it - and I don't plan to any time soon - but it doesn't sound like a terrible idea. At least it's typed and you get a parser for free.

fta

‘’’ But, hey, if you're already paying the price of using protobuf in your projects for some other reason, then why not use it for config storage, too? ‘’’

seems, imo, to be the most important point. otherwise, using it as a config replacement seems suspect.

exactly ! and imo, this is way better than toml :)

This looks much better than JSON. That makes it a winner.

Your comment just reminds me of perfect being the enemy of good.

I’m not sure why you think config files should be self-describing?

I think serialization formats should be self-describing.

If you don't need fancier types or features than what JSON provides you can use JSON for your config and use protobuf to parse it directly.

Used this for a large, semi-generated config file (think large device trees). File was generated once and then there was some light manual editing after. JSON + pb worked great.

Reply on news.ycombinator.com