simonrepp

You are correct! The thinking behind this is that for the majority of file-based configuration and content usecases the expected types are fixed and known beforehand already - ergo it makes more sense that a developer has to specify once which type a field is (gaining in return 100% type safety, validation, localized validation messages, ...) than all users later having to e.g. explicitly write quotes a million times when writing configuration/content, just to tell the application something about the type it already knows anyway (and wouldn't expect/accept any other way too). I think this is really more ergonomic, even in the short run.

tl;dr: Another alternative to YAML (among many great others), this one designed and developed by me:

https://eno-lang.org/

I've been doing a lot of research and development on language design for file-based content (e.g. for static site generators). I've found that YAML - although established as the go-to format for statically generated blogs, etc. - was never designed for these things as it by its nature does not support simple, essential features for this usecase like for instance unindented blocks of verbatim text (for which YAML frontmatter was invented as a very limited hack).

The result of all this R&D is a language called "eno notation" which is designed especially for file-based content usecases, and around which I've also built an entire ecosystem for many languages and editors - if you're working in that field, it might be worth taking a look!

Very cool! If it succeeds and you'd like to publish any part of it at some point, I'll gladly pick it up for a collection of case studies on eno-lang.org, just let me know then.

Meanwhile, I've added a commmunity projects section at https://eno-lang.org/libraries to raise awareness of your Tcl implementation there as well.

Thanks again, looking forward to further developments!

Awesome, very happy to see this.

Now I'm curious - are you putting this to use somewhere already? Or is it for the time being "just" an experiment for the joy of implementing in itself? :)

Thanks for sharing!

The design principle for types in eno is that the language itself has no notion of types, there are only plain textual representations. Meaning, types are never inferred from a document. Instead, when parsing an eno document the application explicitly requests the type it expects for each field, thereby validating the document during parsing. From the perspective of the libraries for parsing eno, types are defined as simple functions/closures (I call them "loaders"), that basically take the string value from a field and validate/transform and return it as the proper type. Some of these loaders currently ship with the libraries for convenience (int, float, datetime, etc. - this is not a hard specification though, just a sensible predefined toolbox), but there can be any number of types, whichever types an application needs in addition can be defined and used on the fly.

This is the main point that differentiates eno from almost all TOML/YAML/JSON like formats, and the implication of this is that eno is extremly simple and very accessible for non-technical users (so far all casestudies have verified this to be true) and a highly reliable data source for developers, but it also comes at the cost of losing generic de/serializability, so it depends on the usecase whether eno should be chosen over other formats. :)

There are also some other things that are unique to eno (e.g. all parser and validation error messages are fully localized and in user-friendly humanized language) or painfully missing in some other formats (e.g. having any number of "multiline string" fields that don't hang indented against an invisible left margin somewhere in the document), feel free to explore further on the website if you're curious!

Thanks for your interest and the feedback on the ABNF (much appreciated!), hope I could clarify things a bit. :)

On npm that's in fact already covered, scrutinize the list https://www.npmjs.com/search?q=eno :)

That road (C or Rust parsing core through bindings) will likely be taken, but for the initial development and jump-starting the ecosystem it was important for me to start with implementations that can be quickly experimented with and iterated and not spend a lot of extra time on dealing with segfaults, memory leaks, the different binding mechanisms on different platforms, etc. As things stand now, people are provided with multiple, fully functioning, pure implementations that already are faster than the majority of YAML/TOML parsers. In the coming months and years there will be plenty of time to make things even faster. :)

For me caring about performance also means caring about performance on all platforms, why not after all? You can take the tabular benchmark data I provide and paste it together, or use the raw data that is also available as eno files in the repository to compare language against language too (which I initially also did but later dropped because same-language comparison for libraries made more sense to me), if you want the quick run down as far as I remember it: mostly javascript parsers lead the ranking, ruby parsers are a bit behind and just slightly ahead of Python.

Thanks for that feedback! I'll see what I can do to communicate the API type concept better, I'm generally struggling to pack all the bandwidth of things into the little available prominent space on the website, but eventually I will get it right for most people I hope. :)

Also thanks for letting me know about the lobste.rs thread, I'll see that I get invited and answer the few not yet-answered points, couldn't get to it yesterday unfortunately amidst all the comment and issue and PR flood here and on github. ;)

Associated fifties right away too, although not that colorful :)

Thanks for the encouragement!

I like the philosophy of friends! (even though I too don't need quite the power of it (yet). ;))

I appreciate the input :) but the thing is that the typing concept in eno as it is now is essentially what makes eno eno. Every application that uses eno decides for itself what types it supports and requires, and that in turn is how eno manages to be so simple and usable on the language level, even for completely non-technical people who normally feel uncomfortable with the idea of editing their content as raw text files.

If I would add types and type inference again, then I would essentially arrive at YAML and TOML again, and I don't want to reinvent them. ;)

But if I actually misunderstood you there, please let me know and do clarify!

Well spotted, good question!

It's also been asked in another thread on HN, I'm quoting myself here: "eno has neither indentation nor closing tags of any sort, that means if you use a section to group some values, you need to start another section to end the previous one (no closing tags!), that's why there are fieldsets, which allow short groupings that automatically end with the next field/list/fieldset."

Hope this explains it :)

Yes through sections! See https://eno-lang.org/introduction/.

You can nest as deeply as you want and multiple sections on the same level automatically turn into a list of sections. For just a list of flat dictionaries you can also use fieldsets, see https://eno-lang.org/advanced/. :)

ruamel.yaml in python does that to a certain degree from what I've read, you might want to check it out if yaml is ok for your usecase! (https://yaml.readthedocs.io/en/latest/)

I've given this some thought as well, and given that the eno libraries hold their own representation of data in memory this might actually be plausible to implement in some way. Still I fear this will turn out to be a hard, hard problem (as eno is not even generically serializable by design), so that's why I haven't explored it further. So for the moment I can only say - Maybe in the near future sometime, check back every once in a while! :)

Sure!

Cultural research = notoriously underfunded, so although they have and rely on a relational database that holds their data (previously Postgres) the cost and effort associated with maintaining and extending the system is pretty high.

With the new setup the thousands of eno files represent both the place of storage and the interface to edit the data, so by that we eliminated the development effort to provide and maintain a full web frontend to the database, and the effort to just maintain the actively deployed technology somewhere and keep it at least patched for security reasons.

All that remains now technology wise is an Atom plugin that is locally installed on each client at the institute and takes care of validating, provides relational autocomplete helpers as demonstrated in [4] and offers a few hooks to kick off local builds for multiple deployment targets and deploy them to live as well.

Hope this clarifies things! :)

Hey, first off thanks! :)

1a) because faster was only one aspect, it also needed to be easier, even more pressingly that in fact. 1b) see answer by other poster (thanks!) 2) Not whitespace sensitive, no way to enter wrong types through syntax mistakes, hardly any learning curve for users because there is so little syntax to memorize, fully localized, hand written parser and validation errors (provided on the library side) ... and so on, check out the website for more, it's all there! ;) Thanks!

Regarding types check out https://eno-lang.org/javascript/#loaders - basically eno allows arbitrary types on the language level and provides loaders for all primitive types and currently also a small set of non-primitive types through the libraries. The extent of loaders provided out of the box might grow, likely also externalized into companion packages like https://github.com/eno-lang/enojs-exploaders/, which currently serves as experimentation ground for this.

The core might in fact be reimplemented in C or Rust and used across implementations through native bindings, although only a small portion of the actual parsing core can actually be outsourced like that, so it will depend what the actual benchmarks say then, there's also cost associated with passing around the data through bindings, the devil's in the details there unfortunately. :)

The editor in the introspection demo is atom, the introspection is based on the excellent automcomplete boilerplate at https://codersblock.com/blog/creating-an-autocomplete-plug-i... paired with a few lines that utilize https://eno-lang.org/javascript/#Section-lookup to determine the exact context for the autocomplete suggestions. Glad you like, thanks for your interest!

One of the design conderations was and is that the format is very strict (and that way predictable), but at the same time as helpful as possible in identifying, communicating and resolving issues.

To that end all error messages that can occur are handwritten, fully localized and shared across all eno libraries (see https://github.com/eno-lang/eno-locales/blob/master/specific...) and the API implicitly handles them for you when you write programs that consume eno.

So basically eno does no magic fallbacks of any sort when faults occur, but it is candid and friendly about it when it happens. :)

From what I saw, I think for at least a few parsers this might be the case because they are built on generated parser code, and it's just easy to run into unfavorable bits and pieces in the output that way, which can drag down performance completely although 95% of the parser are just fine. Technically there's no reasons why toml parsers shouldn't be just as fast or faster as yaml or even eno. :) In any case I'd be happy if the benchmarks stir up some movement and maybe kick off some high performance toml parser intiative, toml is an awesome format and also 0.5.0 was just officially released so there's a good reason to update the parsers now anyway. :)

Conveying what types to enter is not an eno-specific problem, as a user without schema or code access you don't know which types a blank YAML/TOML file expects either!

Asides the absolutely valid meta solutions (e.g. in-file comments, clear key naming, documentation) there is an additional way this is approached in eno: If you use the type loaders provided by the API (say 'my_var = document.url('website')'), and properly expose errors to the user, the user will get a localized (!) error message in his language, like "'website' must be a valid url (e.g. https://eno-lang.org/)".

In the long run we can have community packages for any number of important locally unique types (loaders are just simple functions, so they can be easily authored), so at some point you likely don't have to write any one-off validation code, and neither the error messages or their localizations, you just pull it in as dependencies.

You're right, not yet! Jump-starting the whole ecosystem was a major time investment for me but now that there is public exposure providing a formal spec has a higher priority because someone might actually see it and do something with it ;) Keep an eye on https://github.com/eno-lang/eno, this is where I'm working on it, I'll also announce it on the newsletter (http://eepurl.com/dA9LcH) when it's there!

Syntax vs Semantics is distinguished by ParseError vs ValidationError in the eno libraries - I'll keep the importance of distinguishing them in mind for the schema development too - thanks for pointing this out!

Right now only an ASCII colon is interpreted as an operator, but this looks like a question to thoroughly consider for the next and final spec (which is planned for 2019, currently we're in frozen RC) - work on this currently happens at https://github.com/eno-lang/eno.

There is escaping for arbitrary keys by using backticks - see the advanced language feature documentation at https://eno-lang.org/advanced/, in the case of # #twitter you wouldn't need it though unless you omit the space.

Thanks for your input, appreciate it!

1) Yes! (You can directly dump it to a language-native structure with the raw() method too, this is not 1:1 YAML/TOML style generic deserialization though as there are no fixed types in eno)

2) Some detail aspects of whitespace-parsing around the line continuation syntax will need to be specified by the language, the shared official API I am implementing for the different platforms is fully open to improvement and future reinvention though, I'd love to see a completely new take for a library API if it comes up in the future. :)

3) Definitely!

4) I try to keep things as consistent as possible across the platforms, but if there are important language specific paradigms I think these should be taken advantage of! I can't answer details regarding the PHP implementation yet but keep in touch, I'm happy about a dialogue here! (Also I can't be good at everything :)).

5) I want one! Obviously there can't be a stable generic "just dump it already" implementation, but a smart builder-type API is definitely on the list, I even started one for enojs but had to re-prioritize because there was so much else to do for the whole ecosystem. ;)

section vs. list in eno is like object vs. array in JSON - you need both.

eno has neither indentation nor closing tags of any sort, that means if you use a section to group some values, you need to start another section to end the previous one (no closing tags!), that's why there are fieldsets, which allow short groupings that automatically end with the next field/list/fieldset.

I share your opinion that a single syntax would be the ideal thing, but not having closing tags (which keeps the language simple and fast to write) required a trade-off in the language design to be made.

Why not JSON, Lua table-notation or S-expressions? Because the prime design goal was to achieve greatest possible simplicity and accessibility - almost anyone should be able to use it, no matter the background. If possible I would have wanted eno to be even more reduced and simple, but at some point you have to draw a line too, otherwise you end up with a toy, and then you won't ever get adoption by devs either. So this is why eno ... :) Thanks for your question!

I fully agree with your assessment - eno allows arbitrary types, therefore if and to what extent non-primitive type loaders should be included as core functionality needs to be thoroughly considered and negotiated soon. I included non-primitive loaders (also the exotic lat/lng ;)) to (a) show that this is a possibility and (b) get hands-on insight how well this works in real world usage. (in short: I love it so far, but I'd love to hear other experiences!) It took months to get the whole ecosystem jump-started as a one man show, so that's why some loaders are ... pragmatically coded, you're of course right on that, although admittedly I had no idea the email spec was that complex, thanks for making me aware. ;)

What you're seeing there is actually validation :), the lat/lng type is not magically inferred but instead explicitly requested by the code - if it were not valid it would generate a user-friendly, localized error message. Also the underlying document hierarchy that holds the data is validated. So on the contrary it is actually hard not to validate content in eno.

The quirky looking syntax you mention is probably familiar to you from YAML or paper forms ("Name: Joe"), and Markdown ("# Section"), likewise if you have two levels you use "## Subsection".

There's quite a few things that eno solves (there's only so much space on a frontpage, sorry), but if you want one of the more prominent ones: It's considerably hard to win over users without technical background to switch to secure, statically generated content solutions when the most prominent format works like this: http://yaml.org/spec/1.2/spec.html I've explained eno in 5mins to a non-technical intern who is now managing content at a client of mine and in months I haven't heard a single question about how eno works! User empowerment. <3

Pretty cool, I remember coming across it during research for eno actually! small internet :)

There is considerable shortage of available three-letter file extensions that don't sound like robots from star wars, so unfortunately a sacrifice had to be made there :)

Spot on, that's how it works!

Yes exactly - Hashes denote 'sections' in eno!

In the language design phase this was heavily inspired by markdown, because of the already widely familiar concept of 'headings' - I collected feedback from users that (although they showed considerable anxiety given the prospect that they were to use text files to edit their content in the future instead of a fancy web interface) they found working with markdown (quote) "super easy", so markdown was therefore an influential factor for some aspects of the language design. :)