Hacker News Reader

Comment by 2mol | Hacker News Reader

I'm conflicted between

- this looks very useful, and

- "parse, don't validate" has proven to me to be a far better approach for all my use-cases.

I guess validation that is language/consumer-agnostic is more useful in a big-tech context? Or maybe the advantage is to put constraints on data that you are sending to a system out of your control?

Keen to hear opinions of people that worked with similar tools.

View on HN

Putting constraints on configuration is a big use case. CUE is much nicer to write than Yaml or JSON, and then you can output those formats for the programs which use the "old" formats. There is a vet command if you want to validate Yaml or JSON before moving to generating them. It has helped me catch errors and I use CUE instead of Helm for a lot of things now. I'm looking into generating Terraform JSON rather than HCL. If there was going to be a language, CUE is preferable to HCL.

Another thing I have been working on is a tool to diff, mask, upsert, and transform config or data files in bulk using CUE. https://github.com/hofstadter-io/cuetuls

Do you use Cue for generating config, or for more for validating it.

I kinda like Jsonnet, but it only generates.

True, though one may turn that around to say Cue's scope is too large while Jsonnet's is more "pure" while still enabling the larger scope.

Schema that describes data is itself just data and Jsonnet can describe both. While the various Jsonnet compilers do not "know" about any specific meta-schema in which to express schema nor of course any way to apply that schema to data for validation, one can create such systems with Jsonnet as the language and its compilers as a component.

True, CUE has quite the inspirational scope right in the name. It is actually what draws me to the language. I also like the theoretical foundations in being turing-incomplete. Functions and computation make configuration harder to understand and reason about.

I mostly generate yaml and json now that I've been using CUE. The cool thing about CUE is that it's really doing both at the same time. Types and values are just points along a spectrum of specificity, going from types to constraints to concrete values.

I found CUE because I needed something better than Yaml for generating code, or declarative application code. (https://github.com/hofstadter-io/hof). It was similar to when I found Go and replaced a bunch of C++. I gained way more functionality in my application while shedding more than 50% of my LOC.

The link is broken.

This one?

https://github.com/hofstadter-io/cuetils

Do you also make the cuetorials? It was of great help to us a few months ago. Thank you for that.

I do maintain https://cuetorials.com, so glad you find it helpful! Just trying to share the joy of CUE

thanks for the proper link too

I agree that it’s probably nicer than YAML, but that’s a low bar. I’m more interested in whether it’s nicer than something like Starlark (especially a hypothetical statically typed Starlark).

There is a significant theoretical difference.

CUE aims to be turing-incomplete, so no programming. This is based on Marcel's experience writing borgcfg and GCL while at Google.

The quote I really like is "wrap code in data, not data in code"

It has a lot to do with readability, maintainability, and being able to write tooling to work with config or data. Think about building a query tool that can find everything in your config using a particular label. With CUE, it is likely to be a one-liner (once the query proposal is implemented)

> There is a significant theoretical difference.

I agree, but I don't understand the value of the difference.

> CUE aims to be turing-incomplete, so no programming

You could satisfy this property with an imperative language that lacks recursion, unbounded loops, etc; however, I'm not convinced this property is very useful in practice. I've never worked anywhere where this was a real problem.

> Think about building a query tool that can find everything in your config using a particular label.

In an imperative configuration language, I would just `jq` the generated JSON. At least I think that's analogous to your suggestion?

Marcel does a great job of arguing the value of Turing incompleteness. I've collected his talks here: https://cuetorials.com/videos I cannot say it better than he has.

JQ is close to how CUE would solve this. Both are different than an imperative config language, where you would recurse over data. For both JQ and CUE, you would provide a pattern matching "mask" of sorts. With CUE, you can also have a schema built into that mask and ensure that any transformations are still valid. You can also do the same with Yaml, so you get one tool and language to replace jq, yq, and jsonschema.

Cue might actually get bounded recursion, the idea has been thrown around. It ought not invalidate the theory because it can be proved all the way down. You can simulate it today with comprehensions. I've built some structural helpers on the pattern here: https://github.com/hofstadter-io/cuetils

Can you give more info on how "parse, don't validate" has been helping you? I'm trying to get on the train, but I have so much validating code.

Not OP, but "parse, don't validate"[0] says that data (including config, etc.) should go through a "parsing step" (no "shotgun parsing"[1]) which outputs specific data-types for the main processing code to deal with ("invalid states should be unrepresentable"[2]).

For example, we might have a "parsing step" which takes in Bytes (e.g. '[{"email": "chriswarbo@example.com"}]') and outputs a 'NonEmptyList[ContactDetails]' (e.g. 'NonEmptyList(Email(NonEmptyString('c', "hris"), NonEmptyString('e', "xample.com")), Nil)').

Compare this to a "validate" approach, e.g. parsing the bytes to a 'JsonArray', and using a boolean function to check whether it's empty, and whether its entries match the 'contact details' schema. That's problematic, since (a) it leads to "boolean blindness"[3] (the function tells us 'attempting to get those fields should work', whereas the parsing approach actually gives us those fields) and (b) such a boolean function is actually redundant, since deleting all of its call sites (or forgetting to write them to begin with) would give us a program that still compiles and runs (whereas the parsing approach would give a type error: 'Given JsonArray, expected NonEmptyList[ContactDetails]').

CUE's approach seems to be similar to passing JSON values around, and occasionally performing validation checks (if we remeber to); i.e. validating, not parsing.

[0] https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

[1] http://langsec.org/brucon/ShotgunParsersBruCON.pdf

[2] https://fsharpforfunandprofit.com/posts/designing-with-types...

[3] https://existentialtype.wordpress.com/2011/03/15/boolean-bli...

Its not clear how this complaint applies to a configuration language, since there is no mutability or change, configuration is statically evaluated.

I'd consider cue to be a config language that by and large tries to make invalid states unrepresentable.

Reply on news.ycombinator.com