HOCON – Human-Optimized Config Object Notation | Hacker News Reader

HOCON – Human-Optimized Config Object Notation

github.com

78 pointsEwanToo69 comments

I designed the HOCON format.

Here is my take on the why not JSON / why not YAML questions: https://blog.ometer.com/2015/09/07/json-like-config-a-spectr...

Short answer is "UX"

There's also a historical reason, Play Framework and Akka had both invented ad hoc formats, HOCON replaced both. The ad hoc formats added things like includes and substitutions but in regex-hack type of ways. So HOCON was cleaning those older parsers up with a single actually-documented format.

I've been using HOCON for years.

Why it's not JSON:

* includes

* substitutions

Why it's not YAML:

* less confusing syntax (e.g. `cannot do that` doesn't need quotes, but `can't do that` does)

* built-in support for overrides via env variables

Why it's not JSON or YAML:

* Built in durations! No more head scratching to figure out if 5 is seconds or milliseconds. It's just `5s`.

---

JSON is a data transfer format. YAML is a markup language (or it claims to be...).

HOCON is a composable configuration language. You always need to change, override, and deduplicate configuration in sensible way, in dev, in production, machine-by-machine.

Despite a wart or two I think HOCON takes a sensible to this ubiquitous problem.

YAML isn't a markup language, and it doesn't claim to be. The name stands for YAML Ain't Markup Language.

Except it originally stood for "Yet Another Markup Language"[1], and the newer name only means to separate it from HTML and other document-oriented markup languages.

[1] http://yaml.org/spec/history/2001-08-01.html

Ah, I wasn't aware of the old name, thanks!

Ah, I hadn't heard of the retcon :) It was oirignally Yet Another Markup Language http://yaml.org/spec/history/2001-12-10.html That's why it's YAML, not AAML, BAML, CAML, etc.

In any case, yaml.org currently describes it as "human friendly data serialization".

I don't think YAML is a bad choice, but HOCON's explicit focus on configuration yields some nice benefits.

there's also a little blurb at https://github.com/typesafehub/config#rationale-for-supporte...

What advantages does HOCON have over s-expressions?

Syntax is the paint, semantics is the bikeshed. Every interesting feature of this specification would still be applicable with a sexp syntax.

Agreed, though I think you means to say semantics is the nuclear power plant.

This seems like it'd be easy to shoot yourself in the foot with, especially with the duplicate keys and concatenation behaviour. YAML is also a JSON superset and is significantly easier for humans to parse. This seems to make many of the same mistakes that POSIX shell syntax makes.

This might be an unpopular opinion, but I find it terrifying that people keep repeating the same mistakes. Back in the 90s everyone thought that XML would become the one and only configuration file format. Instead, it became an epitome of over-engineered enterprise software. Now, people are moving to JSON as a serialization format. It's a lighter-weight and usually sufficient distant cousin of XML, but I still wouldn't like to write my config files in it.

The misconception is what might work as a human-readable serialization format is usually tedious as a human-editable configuration format.

Unless your app accepts configuration files as untrusted input, it is really not that hard to write a custom key-value -type parser. Think OpenSSH or Samba config files - I find them well-suited for their respective jobs, as they achieve the required level of expression with minimal necessary complexity.

> it is really not that hard to write a custom key-value -type parser

It's not. But then you want to set up that config automatically. So you template them. Then you discover they have slight differences in what they accept and how they handle unknown/repeated values. Then you discover they have different escaping rules. Then that one needs non-latin characters quoted. etc. etc.

It's not easy to write parser for each one of them. But if you're installing a full server with 10 different config files, you end up with 10 templates which you just hope never run into any edge case of a particular format. And hope that the next service will just use JSON. Or YAML. Or whatever standardised.

Sadly, few people can design a text file format that handles escape sequences properly. I asked this as an interview question for a while, and I've also seen people who should know better get it wrong in production.

It's a really good thing that these days, people use JSON most of the time.

> This seems to make many of the same mistakes that POSIX shell syntax makes.

How's that?

> easy to shoot yourself in the foot with, especially with the duplicate keys and concatenation behaviour

Maybe, but that is the entire point of HOCON: to compose configuration.

E.g. you have a defaults.conf file, but then you have a production.json files that includes defaults.conf and overwrites some of the value.

Or, at the end of your config file, you

    include "local.conf"

and then you can put temporary local changes in a (gitignored) local.conf file.

We use typesafe config for our Java configuration, after inheriting it via Akka + Play. The immutable recursive nature of the API is a slightly bigger win than the config itself, which is just OK.

The way the config composes between reference.conf and application.conf and possibly more confs in the classpath can be a bit confusing, but the API gives you the ability to introspect the source of a config value. It's pretty nifty when you need the overlaying behaviour though.

There's a lot more ports for it available now than when we started using it several years ago, which decreases its previously monolingual risk quite a bit.

We use HOCON for one of our projects and I enjoy working with it. Some of the things I like are:

- It's a superset of JSON - so you can just write JSON if you don't want to change

- The ability to add comments

- Multi-line strings with the use of Scala's triple-quotes

- Reference other values with the use of in-line variables - https://github.com/typesafehub/config/blob/master/HOCON.md#s...

Is it any better than Yaml? All of those statements are true of yaml and it's much more commonly used.

The two differences from YAML that seem obvious are "includes" which reference other files, and drawing values in a HOCON file from an environment variable.

Both to me seem easily encoded using YAML's support for custom types.

It would be useful to show some concrete examples instead of just documentation.

Exactly my response. A side by side showing an example of JSON and its HOCON equivalent would be immensely useful

Here are some examples

https://github.com/typesafehub/config#examples-of-hocon

Here was my use case:

We had an existing config system built around json that was typically machine written from human GUI manipulations.

Our existing system could process json but not HOCON.

HOCON after it parses config file(s) provides a tree that can be written out as json.

I had to I start writing the configs out by hand instead of using the GUI.

So I started refactoring my config as HOCON because it has some moderate boilerplate reduction and DRY helping features. Then I'd load those new configs with the HOCON library in Scala and have it spit out json which the existing system could then parse (in memory did have to write out files fortunately).

I was in a Scala stack so using HOCON was a natural fit. I tried snake-yaml as well first but HOCON was easier for me to get up and I felt had a less wonky syntax. (I use yaml quite a bit for ansible so I am fairly used to it but even ansible extends yaml a bit to make it easier to write).

Writing configs in json is a really big pain because:

1. No comments

2. No comments

3. "Having" "to" "quote" "all" "the" "keys" "and" "strings" "is" "painful"

I understand that perhaps I could relax those rules in the json parser but then I am just a step away from HOCON which adds higher order stuff.

IntelliJ had good HOCON support (ctrl+click to go to key / error highlighting) too so that helped. N.B. their yaml support is also good (via a plugin).

Anyhow keep in mind the way I used HOCON was quite different from how I have seen it used by other devs in the space. Whereas they included it in their jars then let users override stuff at runtime. I was able to use some of those design goals to make a modular slightly higher order json config system where I could share common configs as importable overrideable libraries.

Now if I had to have a system where humans and machines are "editing" the same file which was a case I had considered supporting for my system, I think I'd try to move the problem away from the same file space. And ask the question: Is there any way we can keep them out of the same files? Because that is probably easier than having the machine side keep track of comments in the parse tree and write them out when changing stuff. I'd try to sidestep by having my config system support importing files and merging them together and look more at how the directory.d pattern works in linux.

I noticed a few people here doing double takes at HOCON, which I did as well (let's say it's a good instinct). After using it, things made a lot more sense.

* Old school Gnome users may recognize its original author Havoc Pennington.

* Although it's a Java library, it's more natural in a Scala; the config objects are immutable, and Scala people like to rewrite the whole world anyway

* It's a very nice way to organize and write configuration files, although for our team we needed a session to understand how the final configuration was being derived since there can be multiple configuration files as well as the fallback/default values in reference.conf.

Json is completely fine. It's just too bad it doesn't have any support for comments. Maybe a future revision could add it?

Well, not for everything. If you're working with scientific data, the lack of NaN and Inf can be problematic.

Not that HOCON solves that.

'inf' and 'nan' should work fine. If I understand your suggestion correctly, you would like ieee floating point inf, -inf, and nan to be representable as tokens as though they are numbers. It makes sense as that's how ieee floats do it. But in JSON they aren't numbers, so they aren't represented as numbers.

From the spec:

""" JSON is agnostic about numbers. In any programming language, there can be a variety of number types of various capacities and complements, fixed or floating, binary or decimal. That can make interchange between different programming languages difficult. JSON instead offers only the representation of numbers that humans use: a sequence of digits. All programming languages know how to make sense of digit sequences even if they disagree on internal representations. That is enough to allow interchange. """

I'm not suggesting anything, I'm fully aware of the spec. I'm just pointing out that you can't represent NaN and Inf in JSON. Strings don't count.

This has been around for a very long time. It's standard for configuration in basically all scala projects.

It's current year. Why haven't the rest of you realized that commas should be whitespace?

Specifically, tabs.

Commas are explicit, whitespace is implicit. Whitespace wouldn't even save characters, because it requires either a tab or a space.

Here is a comparison of json like formats by the author of HOCON: https://blog.ometer.com/2015/09/07/json-like-config-a-spectr...

So when might I prefer this over YAML or TOML?

Used this recently with Akka.NET

It took me back to the days writing reams and reams of xml for JBoss back in the day.

config is code and I see no compelling reason to adopt HOCON over code, or other external DSLs like YML, TOML, JSON and XML.

How do you deal with compiled languages, eg native or JVM ones, that you want to deploy and configure at runtime?

Configs let you have a container/binary/deliverable that you can test/tag/workflow on as an artifact.

I've always felt that YML, TOML, JSON and XML cover my needs.

There are other configs like docker file, but they're vender specific.

How does it compare to yaml?

Yaml parses with this. You can write yaml but use the features of hocon like variables and includes

Yaml has something similar to variables with its concept of anchors.

To me YAML is the ideal human readable structured data representation format, at least for humans with sufficient visual acuity to pickup whitespaces. I've heard people hate it with a passion, but I could never get them to justify said hate except to say that it's "ugly", "unmanageable" or "primitive".

The YAML spec is a ridiculous hodge podge. It's a factory farm packed with feeping creatures.

The kind of person who cares about that sort of thing is the kind of person who hates YAML passionately.

It sadly is, but the core syntax of YAML IMHO is still nicer than any other. To that end, I wrote a parser that simply cuts out the crappy features leaving just the basic core (and added optional type-safety/validation):

https://github.com/crdoconnor/strictyaml

I also tried to write up a more extensive justification for anybody who scratching their head trying to decide which config format to use (including a "why not HOCON" paragraph):

https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst

Nice project! I agree about the YAML core syntax.

And your FAQ is so well argued that I submitted it to HN: https://news.ycombinator.com/item?id=14318472

Thanks!

I don't particularly like YAML because whenever I see something written in YAML, it's not completely obvious to me how to modify it.

When I see JSON, I can sort of remind myself of the syntax: OK, things are comma separated, braces are matched, keys/strings are quoted, and that's it.

When I see YAML, I have to open some manual page with the syntax. (Ditto for anything with "vague" syntax like Wiki, Markdown, etc.)

I personally think that "vague" syntax is a bad thing in computer languages, and it's better to keep syntax to rather obvious minimum. Like in s-expressions, for example.

Are there libraries for HOCON written in other languages?

I wonder if the authors are aware of http://hjson.org/. It has similar features as far as loosening the json syntax rules and a lot of client libraries. HOCON seems to go a lot further in features/complexity.

https://blog.ometer.com/2015/09/07/json-like-config-a-spectr...

I wrote HOCON and also the above comparison post.

Fwiw, briefly looking at the tags in the typesafehub/config repo and the "history" section of hjson.org, it looks like Typesafe Config predates hjson by several years.

re: other languages: yes, there's a list of implementations in the typesafe config README

I like git's config notation. Don't know if it supports references though, and other general "object" characteristics, like explicit types for polymorphism.

Then again, Java Script Object Notation supports neither.

This is interesting, but I still find HJSON a better "JSON for humans format". HOCON seems to mix in things that better belong in the consumer, not the objection notation format.

I would say HOCON isn't JSON for humans, it's config file for humans. The features are all about building up a config in a DRY way and merging overrides with defaults and overriding with env vars and stuff like that. It's specifically for the config use case.

JSON superset is a bonus feature which is useful if you don't want to learn new syntax or want to machine-generate part of your config.

seems similar in goals to HCL https://github.com/hashicorp/hcl but harder to write.

I use HOCON at work, tripped up today because linebreaks matter! (no way to do a delimmited comment)

something has to matter :-) after making commas, braces, brackets, and quotes optional linebreaks were what's left...

Is there a meaningful difference between linebreaks and other whitespace if you have no comments?

I understand that it might constitute a breaking change now to delimit the comments since:

# HOCON is great you can have #comments!

would now be a compile error but I'm interested if there's another issue that prevented optionally delimited comments in the original spec.

This is the sort of thing nobody needs or wants at this point in time.

WOW, I didn't know that you could speak for the ENTIRE WORLD.

Personally, I find this at the very least interesting and possibly useful. The lack of comments in JSON is a problem when it comes to annotating configuration stored as JSON. But we have other formats that do not have this limitation (INI, YAML are two examples).

I really like the idea of "Human-Optimized" config, because it is more important for code and config to be understood by humans than by machines.

We're using it for distributed app configuration management at $work because Typesafe accepts it.

No complaints about hocon itself, but Typesafe configs get applied in first-value-wins which is a continual source of ... amusement ... for developers.

Wow, first value wins seems like a really horrible idea, any insight into why that choice got made?

There seems to be a confusion. First value wins - is in no way how hocon works. Just take a look at the examples

I'm pretty sure the last value wins, but maybe don't understand what you mean.

Available evidence indicates otherwise.

Watch: 168 Star: 2,753 Fork: 429

Ah, I may have misinterpreted the intent. That helps, thanks

It's cool, I suppose, but I don't think I would ever use it. If I need something more complicated than a basic ini, json or xml file, I think I'd reach for a scripting language I already have in my toolbox, like Python or Lua, or JavaScript, than learn Yet Another Configuration Language.

I'm with douche. If your config is so complicated you need variable substitution and regex then it's no-longer config: it's a driver program. Might as well program in a programming language.

So, when I first learned about YAML, I thought it was a great idea. I was wrong. I've been burned too many times by ambiguity. Config file formats should have a grammar that fits on a sheet of paper. Maybe two sheets.

I think this is reasonable in some contexts but it depends. Often the configuration is done by a different person or in a different context than the development. The configure-er may or may not be a programmer and the deployment environment may or may not have xyz programming language available, etc.

The JSON superset aspect nicely supports spitting out JSON from a script when people want to do it that way.

HOCON is nice when for example you just want to sub in a couple of env variables or avoid a little cut-and-paste.

Reply on news.ycombinator.com