Preserves: An Expressive Data Language | Hacker News Reader

Preserves: An Expressive Data Language

gitlab.com

22 pointstosh16 comments

Lots of deep technical detail, but where’s the leading sales pitch? Tell us what real-world Needs and Wants drove you to create it in the first place, and how your solution fits these problems better than popular established rivals.

Thanks. That part isn't yet written. (It's early days still.)

In a nutshell though: JSON (for example) doesn't mean anything. It has no semantics - it's just syntax. I wanted something that had a semantics. Plus, proper support for binary, records, and sets, and keys in dictionaries that didn't have to be strings.

There's a wee proto-essay on "why not just use JSON" in an appendix: https://gitlab.com/tonyg/preserves/blob/master/preserves.md#...

There's also a note on why not SPKI Sexps here: https://gitlab.com/tonyg/preserves/blob/master/preserves.md#..., but it doesn't render in the rendered markdown because there's a bug in Gitlab's markdown renderer: https://github.com/github/cmark-gfm/issues/121 (see also https://gitlab.com/gitlab-org/gitlab-ce/issues/52013)

Honestly, “Why?” is the first, not last, question you need to answer, not least to help clarify your own thinking and nail down your end-goals. Who’s its market, and what problems do they have that this solves? Otherwise it’s just one more solution in search of a problem, going nowhere much.

Oh shit. I wasn't really ready for this to appear here so soon. That said, comments and feedback welcome!

If I understood right annotations are part of the parsed data? Are they idempotent then? I know some people maintain that you can add comments in JSON but I hate that they disappear on a deserialize/serialize unlike XML comments.

The idea is that in most programs, you'd write code as if the annotations were not there. Annotations should, as a rule of thumb, not affect control flow of your programs at all.

But implementations should let programs access the annotations if the program explicitly asks to do so.

This lets us write "ordinary" programs, where the annotations are transparent, and "meta-level" programs, where the annotations actually do something - think IDEs, debuggers, message-tracing-annotators, provenance-annotators, etc.

The equivalence over values is defined so that annotations are ignored for comparison purposes. So `{ @a "a": 1 }` is equal to `{ "a": 1 }`.

[ETA] Implementations should take care to preserve annotations during round-trips; but to do so in such a way that "ordinary" programs don't even notice they're there.

Finally, annotations are the most "experimental" part of the design at the moment, so feedback on annotations in particular is most welcome.

In the encoding examples, I would prefer showing ASCII characters that encode themselves as literals. So for the "hello" (format B) example, which is currently shown as:

    55 68 65 6C 6C 6F

it would make sense to me to display it as something like:

    <55> h e l l o

    55 <h> <e> <l> <l> <o>

depending on which version you prefer.

Good idea, thanks, will do.

This is a super interesting project. Have you considered a derivative project that would add evaluation and functions?

Yes, but it's even less ready for public scrutiny :-)

Actually, could you say more about what you have in mind? Something a bit like a config language, maybe -- terminating, sub-Turing -- or something more like a Lisp?

Something like a Lisp yeah. I think sub-Turing is interesting for verification/configuration/whatever, but I have a soft spot for small languages with powerful base concepts.

Now you guys made me think of https://github.com/Gabriel439/Haskell-Annah-Library again.

I applaude the author to think about sum types in the data language but the semantics part is slightly confusing. First it tells me that all records are equivalent and later it tells me how they are compared against each other. I think there should be a distinction between the form and their content being equivalent.

Thanks for the feedback. However, I don't understand quite what you're getting at. Can you point out the wording you find confusing? I don't see anywhere that it says all records are equivalent.

Re: distinction between form and content, that's the intended aim of having separate "semantics" and (two) "syntax" sections. The idea is that to compare Values (i.e. to know which Value you have!), you work in terms of the "semantics" section, not in terms of any particular surface syntax.

You write

> Equivalence. Two Values are equal if neither is less than the other according to the total order.

Directly below the total order of what you call "kinds". That confused me.

Thanks, I'll try to clarify.

Reply on news.ycombinator.com