Ron: Rusty Object Notation | Hacker News Reader

Ron: Rusty Object Notation

github.com

123 pointsingve125 comments

Nothing will ever unseat JSON because whilst it's crap, it's good enough.

The toughest competition in the universe is a "good enough" competitor.

Startups competing against an existing, good enough competitor have this problem too.

JSON is fine for inter-machine communication (like APIs). It's IMO terrible for human-to-machine communication (like configs), which is why there are so many alternatives trying to improve in this space (TOML, YAML, RON, etc.).

For inter-machine communication TLV binary formats are far superior. Simplicity of parsing, efficiency. Can't compete.

JSON still wins because everything already has a JSON parser.

Only when various performance metrics are not important enough to consider alternatives.

If performance matters, TLV doesn't cut it either, you need fixed-offset fields. Self-describing binary formats like CBOR, BSON, bencode, DER, UBJSON, etc fall between two stools, being less hackable than JSON and slower than schema-driven binary formats, which is why they are all moribund today.

Everything has JSON because everything has a web browser with JS these days. Calculation can change if browsers stop shipping or stop shipping with JS.

Dhall is the most interesting alternative because it can be compiled to JSON easily

That's like the least interesting feature from Dhall, and also probably one of the most common features of any new format. Not sure I could mention any even semi-popular format that cannot easily be converted to JSON.

https://dhall-lang.org/

Cool indeed. I wonder how you can supply it with types from the program that consumes the config.

My Gradle config is in Kotlin these days. Kotlin, besides being a full blown prog lang, has nice features for config specs (map/list literals, typed, eDSL syntax). Though it is an enormous dependency (way to big for a project that just needs a config file format).

The bigger Dhall type libraries are usually generated from OpenAPI specs - there is a core package that consumes them.

If your language has an OpenAPI library, you can create a hello-world web app that exposes your configuration object as an HTTP endpoint, and generate Dhall types from that.

After trying both Dhall and Cuelang, I really prefer the latter.

But this turns out to support OP's point. JSON is good enough, and there are many alternatives that we are debating and don't agree on.

So if you have to pick something, and I have to pick something, since we don't agree, we will fallback on JSON.

man, nailed that

Worse is better, yet again.

This may be right for a big distributed open source project or something, but most things I work on are proprietary within companies, and in that case it's easy to just pick something better and standardize on that. My current company is pretty much standardized on yaml, which isn't my favorite, but it's fine and better than json and we don't argue about it much. Things get serialized to json pretty often, but most stuff that is checked in and intended to be human readable is not json. A different company could easily pick cue or whatever else and it would work just as well.

I think this is a neglected point. For application config, the scope over which standardisation pays off is smaller than for data. It can be company-sized. Or language-sized - if all Ruby library use YAML, that's great for Rubyists; if all Java apps use properties files, that's, believe it nor not, pretty great for Java programmers.

Huh, yaml is just all around the worst on any metric. I would much rather stick to xml even.

Like I said, yaml isn't my favorite either, but this is silly hyperbole; xml is way harder for a human to read and write.

JSON is perfectly human readable. Anyone claiming otherwise hasn't seen a binary format.

Where JSON mainly fails as config format is lack of comments. Comment provide help and structure when editing. While allowing to toggle functionality. Which is where trailing commas help.

Binary format is perfectly human readable. Anyone claiming otherwise hasn't seen an encrypted binary format.

Encrypted binary format is perfectly human readable. It's not my fault you weren't born a savant. Anyone claiming otherwise hasn't seen white noise. /cj

Lets spend months replacing a thing that actually currently works with a slightly nicer to read file format with new and exciting bugs?

Use a prettifier plugin in your editor or squint, silly human. Trailing-comma-on-write-annoyance-adjuster, bada-bing, bada-boom. Most of the peeves can be hidden with good tooling. This Ron thing makes sense because of Serde not because it is prettier.

Let's not forget KDL! : https://kdl.dev/

HOCON is a great human-readable alternative to JSON. It's a superset of JSON with lots of cool features that make it both more readable and easier to use.

Here's a rundown of HOCON's main features: https://github.com/lightbend/config#features-of-hocon

I have had great experience with HOCON. I miss it in non-JVM languages.

I would say I've mostly landed on JSON by default and TOML for configs when possible. Otherwise, that ends up being JSON too.

> Nothing will ever unseat JSON because whilst it's crap, it's good enough.

JSON is "secretly" being replaced by JSON5 in quite a lot of contexts. So I'm not sure that this is entirely accurate. The same was said about XML a long time ago and people claimed that XML will never be replaced. Yet today XML is barely used any more. Likewise TOML is showing up in more and more places were previously INI or YAML was the standard.

Of course, all of this depends on your context. It's possible that one's world is all about JSON5, while another person's world is all about XML (yes, still). It all depends on where you work, what FOSS you contribute to, what code you write, what language you use.

TOML for example is extensively used in the Rust world, but if you're doing PHP, you're much more likely to deal with XML or JSON than TOML on a daily basis.

XML had an aura of inevitability, but everyone hated it. That’s why JSON won, it isn’t vile like XML

XML would have been so much more popular if it hadn't had A: namespaces and B: just about the worst imaginable syntax for same.

There is no end-state for formats where one "won" and others "lost". XML "won" in terms of it being used everywhere in the past, JSON "won" in terms of it being used everywhere right now and some other format will have "won" in terms of it being used everywhere in the future.

Formats come and go over time, surely JSON will as well.

JSON5 looks great! https://json5.org/

I wish. But not only this is not well supported, the parsers are usually very slow.

I have never come across anything that actually uses JSON5.

Postgraphile comes immediately to my mind. https://www.graphile.org/postgraphile/smart-tags-file/

JSON5 parsing is unfortunately slower than JSON. Not intrinsically, but JSON.parse/stringify are both natively implemented and highly optimized. If browsers started supporting it, it would take off overnight. I can't tell you how often I've wanted to put comments or not worry about removing the last trailing comma from a list.

It would win for some of the same reasons HTML5 beat XHTML. Ease of use and implicit rules beat pedantic strictness any day of the week. "Programmers are lazy."

You would be surprised how widespread it is across certain ecosystems. Even the apple JSON parser supports it. A lot of the .json config files in the JS/VSCode ecosystem are actually JSON5 files.

JSON5 is specifically for hand-written config files. So it can only be replacing JSON in that context, or?

I didn't know JSON5 existed until this post and from now on will use it everywhere.

Kudos to Asheem and all JSON5 team for building a great product while ignoring the haters [1]

[1] https://aseemk.substack.com/p/ignore-the-f-ing-haters-json5

That's why open source is not enough anymore, we need _lean_ open source, and that stable in time.

Big Tech is based mostly on 2 related scams: planned obsolescence and grotesquely and absurdely massive and complex open source software/standards/protocols.

To protect us against them, is to go to simple but able to do a good enough job and stable in time protocol/standard/open source software/etc. You will need lawyers.

This is common sense, but many seem to lose it... more or less sincerely...

I wouldn't be so sure about that, I thought the same about XML in 2005...

Thinking about it, it has 1% to do with the format at 99% to do with various lock-in mechanisms. The explosion of the web and JSON being the standard for JS definitely helped it alot! Protobuf has done pretty well on the backend based on the "won't get fired for copying Google". XML had it's own lock-ins, being the basis of XHTML at that time, and Microsoft had a big fetish for SOAP.

While I think (and lament) that you are almost right -- that is, that ridding the world of the foul pestilence that is JSON it will take insanely longer than it should -- I do think it will come a bit sooner than the heat death of the universe.

The reason I think so is that so many JSON users are no longer actually using JSON. Although there are apparently several competing implementations still vying for the name "JSONC", one of those is the non-JSON format that allows comments used by VS Code (among many others), and it is spreading (often inadvertently).

This weakens JSON by diluting the meaning of the term (many users of this "JSON plus comments" think they are using JSON). The end result might be that we standardize on some "JSON+" type name, or that the meaning of "JSON" itself is diluted to include such superior mutations.

Either way is pain in the ass, though, because JSON is a human-readable format (I mean, not really, but in practice), but it is also a machine-to-machine format, so deviation from the spec is certainly unhelpful there.

But we got rid of Latin-1 and SJIS (for the most part), and I have hope that in the fullness of time, we'll get rid of JSON, too. (Probably not with RON, though.)

P.S. Nrwl guys, JSON config files are literally the most painful part of using Nx, and you should consider just using TypeScript, but if not, "VS Code style comments-enhanced JSON" would still be a massive improvement. :-D

They thought that about XML, and ASN.1, and...

Agreed with this general notion however, I'd argue that unlike a standards body, a big driver for JSON's ascent to defacto standard were the legions of javascript developers working at every layer of a tech stack using it as a low-friction means to an end.

I think it will for config, simply because JSON doesn't support comments and that's really not an optional feature.

But I have tried lots of alternatives and none have as good tooling except maybe XML. Especially for schemas, which I think is really important.

I want JSON5 to succeed but there's no good IDE support really.

I also tried Dhall, which looks promising except they've imported all the super weird stuff from functional languages that are just going to put people off (e.g. the weird leading commas, and very odd way of declaring functions).

I don't see this intended to unseat JSON. But it can be helpful for local testing and debugging of Rust projects. I wouldn't really expect this for general purpose serialization because JSON is more standard and for inner-app communication you can use something more compact.

I did actually use this in a project because it differentiates between nesting of Option. Particularly we were having some confusion of Some(None) and None and this helped our tests catch more issues. In production we used JSON. Serde makes it trivial for the same code to switch between formats.

I just wrote a little cli that takes JSON input, deserializes it into rust structs, serializes it into my custom binary protocol, sends it to my server I'm developing, and then does the reverse with the reply.

This looks like a nice replacement because it looks like the debug logs of my types. Plus it lets me specify numbers as hex.

We’re using something similar(ish) internally and it’s interesting how much more readable and expressive JSON can get by allowing Rust style constructors/tags for your data types.

Suddenly sum types become conveniently easy to express and understand, which is always incredibly awkward in JSON.

Besides that: tuples, tuple structs, comments, properly handling NaN, optional.

Really hope something like this takes over the role of JSON/yaml soon.

Can you show examples? What you say is intriguing.

What's worng with YAML? It seems to be the best of the kind I have seen.

What isn't wrong with YAML?

But first a quiz of two YAML documents.

Yaml#1

Yaml#2

Is any of these two docs valid. If yes, why, if not, why not.

Edit: To be succinct (assuming Yaml 1.2), what is wrong with YAML are string types, weird quoting rules, attempts to be JSON, really bizzare user friendliness rules causing really bizzare edge cases, lookahead & lookbehind being essential for Yaml, very context sensitive, and complicated to the Xth degree (where X approaches infinity).

Yaml feels like it can't decide whether it wants to be whitespace sensitive (human readable but not editable) or whitespace insensitive (mostly human readable and human editable).

9 hours and no solution. And this is second time I post this.

Goes to show how surprising YAML is, which is it's worst property.

https://noyaml.com/

A big addition that json is missing a distinction for, which Ron has is the notion that hashmap and record/struct type are different. Hashmaps, you don't necessarily know what all the keys are. Structs, you know the keys and what type each key should be.

Moreso that semantically hashmaps are collections and structs arent. Structs are single items with a fixed number of properties.

hashmap's a collection type, record/struct is a product type (tuple is another product type), many miss the sum type (incl JSON) sadly.

JS conflating the hashmap and record/struct is a big loss, probably fixed by TS. it does show JS's dedication to its radical dynamic typing discipline (or lack thereof).

Ron is quite practical if you are having a serializable interface, and you want to get debug like output from it for snapshotting purposes. I am using this with my insta snapshot testing library (https://insta.rs/) in some projects.

Sweet, it supports trailing commas! Single biggest personal JSON annoyance.

That, and comments!

Personally, I really hope Human JSON, https://github.com/tailscale/hujson , will get more popular!

There's also JSON5

https://json5.org

JSON initially had comments.

https://www.freecodecamp.org/news/comments-in-json/

They were removed from the standard because some were using them for custom parser directives.

I feel like that is throwing away all the advantages of comments with any benefits. If somebody wants to add parsing directives, they could still be added through specific keys or strings.

That's a big pet peeve for me as well! Only thing remaining now is those pesky colons (edit: was semi-colons) that everyone and their mother seems to place in their new formats. Just get rid of them already!

Who wants to type `{name: "pveierland"}` when you could just do `{name "pveierland"}`?!

Not a fan, it looks kinda alright when you are using a string like that, but {num 5, val 4} looks alien to me. If you used this style in a programming language you'd even get {num var} which is even worse.

Alien is not a reason something is bad, just that's it's unusual. JSON was a bit alien when it first arrived as well, as everyone was used to XML at the time.

`{num 5, val 4}` looks fine to me, but we can do even better! We already know objects/maps are always in pairs, so we don't really need that comma either. Just do `{num 5 val 4}` and we save yet another unnecessary characters.

Of course, I didn't come up with this format myself, what I actually want JSON to be is EDN (https://github.com/edn-format/edn) which is a standalone format but also directly used in Clojure, so it already exists inside a programming language and works very well. There keys are strings though, so you example would end up being `{"num" 5 "val" 5 "person" var}`, where commas are optional.

For machine to machine interchange formats, removing redundant elements is reasonable, but for formats that are meant to be read (and more critically, edited) by humans it is better to have syntactical redundancy. It makes it easier to scan documents and much, much easier to provide human-friendly parser errors when we inevitably make mistakes. The redundancy works as flagposts that both machines and humans can use to get their bearings even in the face of malformed input.

I think it's highly subjective what's easier for humans to read/write.

Personally, I much prefer the latter of these two:

JSON:

    [
      {
          "id":"0003",
          "type":"donut",
          "name":"Old Fashioned",
          "ppu":0.55,
          "batters":{
            "batter":[
                {
                  "id":"1001",
                  "type":"Regular"
                }
            ]
          },
          "topping":[
            {
                "id":"5004",
                "type":"Maple"
            }
          ]
      }
    ]

EDN

    [{"id" "0003"
      "type" "donut"
      "name" "Old Fashioned"
      "ppu" 0.55
      "batters" {"batter" [{"id" "1001"
                            "type" "Regular"}]}
      "topping" [{"id" "5004"
                  "type" "Maple"}]}]

Strings (like `"id"`) are usually keywords (like `:id`) in EDN, but I still find it much easier to parse and understand. I've read a lot more JSON over the years than I've read EDN, so I don't think it's a "used to" feeling I have either.

Oops, I accidentally selected a couple of bits and hit delete…

EDN

  [{     "0003"
    "type" "donut"
    "name" "Old Fashioned"
    "ppu" 0.55
    "batters" {"batter" [{"id" "1001"
                          "type" "Regular"}]}
    "topping"
                                }]

This issue occurs even if you use the comma syntax:

  {:id 3, :type, :ppu 0.55, :topping}

parses to:

  {
    :id 3,
    :type :ppu,
    0.55 :topping,
  }

The worst thing a human-readable data format can have is syntax that looks like it's redundant (and capable of error-detection), but is actually ignored.

What if we could have minimal redundancy and still be capable of error detection?

Check this[0] out:

    [
        id [0003]
        type [donut]
        name [Old Fashioned]
        ppu [0.55]
        batters [
            batter [
                [
                    id [1001]
                    type [Regular]
                ]
            ]
        ]
        topping [
            [
                id [5004]
                type [Maple]
            ]
        ]
    ]

Now if we delete `id` we will get a syntax error. And yet no commas, no colons, no quotemarks! Only square brackets. Minimal redundancy.

For a bit more error-checking-thru-redundancy we could analyze indentation (one reason why I recommend C-style rather than Lisp-style formatting) and warn if we detect any inconsistencies.

Ergo: through design magic we can get rid of a lot of the redundancy and increase desirable properties, without trading off much of the positive side.

NB for a machine we could compact the above into:

    [id[0003]type[donut]name[Old Fashioned]ppu[0.55]batters[batter[[id[1001]type[Regular]]]]topping[[id[5004]type[Maple]]]]

[0] More on that here: https://news.ycombinator.com/item?id=35675811

If you actually want error-correction, I don't think either JSON or EDN is suitable for providing that out of the box. You probably want to provide error-correction out of band in that case.

With your argument, JSON is equally bad because you could have {"name": "wizzwizz4", "age": 16} and ooops, some things got deleted and now the structure ends up being {"name": 16} which isn't really a sound argument against the format itself.

With JSON, few typos go undetected, and an (undetected) local typo causes a local error in the data. With EDN, many typos go undetected, and a local typo can cause non-local errors.

Another situation, due to the same defect:

  {:name Jason Bourne, :age 320 000}

The JSON-equivalent mistake would be a syntax error:

  {"name": "Jason Bourne", "age": 320 000}

That's a colon, not a semicolon. I prefer them myself, as I find them easier to read and slightly simpler to process in ad hoc ways with sed etc.

Cheers, I'll blame the typo on lack of coffee. Although I'd like to get rid of both colons and semi-colons.

Linked from elsewhere in this thread: KDL

From the repo:

> Note the following advantages of RON over JSON:

>trailing commas allowed

>single- and multi-line comments

>field names aren't quoted, so it's less verbose

>optional struct names improve readability

>enums are supported (and less verbose than their JSON representation)

I’m no fan of JSON but honestly, I think this is a very weak list of benefits if you’re trying to convince someone to switch.

Agreed. I don't know why "trailing commas allowed" is a plus -- it is not going to make things simpler or easy to maintain, and it is very questionable whether the added flexibility is worth it

Huh? It reduces the logic needed to write the format.

The big reason for potential users is likely this one:

> It's designed to support all of Serde's data model

As long as we are on topic of JSON alternatives suitable for configuration, I've been tinkering with various minimal syntaxes and formats for a long time, most notably Jevko[0].

One format based on that I've been conjuring up recently would represent the first example from the RON README like so:

    GameConfig[ optional struct name
        window size [[800] [600]]
        window title [PAC-MAN]
        fullscreen [false]

        mouse sensitivity [1.4]
        key bindings [
            up [Up]
            down [Down]
            left [Left]
            right [Right]
            
            Uncomment to enable WASD controls
            ;[
            W [Up]
            A [Down]
            S [Left]
            D [Right]
            ]
        ]
        
        difficulty options [
            start difficulty [Easy]
            adaptive [false]
        ]
    ]

I put a syntax-highlighted version and some more details in this gist[1].

I wonder what you guys think about such a minimal alternative.

[0] https://jevko.org

[1] https://gist.github.com/djedr/4eeac1de466512ec211ff17cfd1f5e...

Why have mandatory commas at all? Both Ron and JSON can be parsed unambiguously without commas.

I am using RON in a gamedev experiment I'm doing and it's great for live reloading of stuff

An Object notation for every language? Perhaps this is the way all languages will go but it seems a little bit retrograde.

e.g. Python Object Oriented Notation (POON) would have double quotes and True and False and "None" but otherwise be easily mistaken for JSON.

Scheme Object Notation Externtion (SCONE) would presumably just be a lot of brackets?

"Scheme object notation" is called s-expressions :)

I'd argue that toml can be called "a Python(ic) object notation" in some sense.

you can't cut and paste all of TOML into a python program though. You can almost cut and paste JSON though - I think that's closer.

I wish this were more commonly used - just having comments is so useful!

Are there already parsers available for Scala, Python, Java, etc.?

The biggest advantage of json is its ease-of-use. It is super easy to get started, and once you are start using it to build a quick prototype, you are stuck with it.

And the customers of your api understand it as well, so you likely need to support json even if you may use something else internally.

Bookmarked just because I like the name.... ;-D

So all JSON is valid (modern) JavaScript. [1]

Is RON valid Rust?

[1] https://github.com/tc39/proposal-json-superset

The first three advantages are present in javascript, and removed for JSON:

    trailing commas allowed
    single- and multi-line comments
    field names aren't quoted, so it's less verbose

Those advantages also apply to lua tables, fwiw.

Looks neat but honestly I think I'll stick with toml for config files

Can it parse Rust debug output (dbg!() or {:?}) back into original data?

Usually not, because Rust's debug output is largely unstructured. There are some projects that are trying to do it though: https://crates.io/crates/darrentsung_debug_parser

It’s not intended to do that, and I don’t think that would be possible in general. Debug output can be arbitrary and ambiguous, while this is a well-defined serialization format that only happens to have a readable Rust-like syntax. Of course you’re always free to use RON in place of debug output for logging purposes.

I could output `ksdjflaskjdflkajsdf` every time my custom structure is Display'd or Debug'd. There's no sense in trying to parse that output.

Who needs TOML when we have this?

Also this is really cool when you want to upgrade your config file to a script!

It has comments! (looking at you JSON; pre-5 I know...)

Is it yet another markup language?

That's YAML. This is object notation.

Yes, it was a tongue-in-cheek reference to YAML.

"Markup languages" are for documents. RON, YAML and JSON are data-oriented.

Even though YAML was named "Yet Another Markup Language" in the beginning, it changed to "Yaml Aint a Markup Language" to emphasise that.

What are the security implications? It sounds like they might be feeding the config text directly into Serde - which means allowing potentially untrusted user input to instantiate arbitrary structs. That's a big attack surface.

None. Serde is strongly, strictly typed: you have to specify what type you want to decode to. It’s nothing like Python’s Pickle protocol.

See, for example, https://github.com/ron-rs/ron/blob/484fcab0686dfd18c7e29b6c1..., where it (in a type-inferency way) says “parse as Config”.

I appreciate the response, but the link you gave only states that the value returned by from_reader is strongly typed; it says nothing about what from_reader might be doing internally before it returns that value.

The actual answer seems to be https://github.com/serde-rs/serde/issues/1087#issuecomment-3... - no security implications if no manually implemented Deserialize impls linked to by your application have security implications.

Well that’s just common sense. You write the deserialiser, it’s your responsibility. It’s no security implication in Serde or Ron.

The interesting part is that it's also your responsibility to audit deserializer code in your dependencies.

That’s still just a normal aspect of code. If you care about security, you have to audit all your dependencies.

Security concerns in deserialising come about due to the input being able to control things, things like Python’s Pickle protocol where you get arbitrary object instantiation and function calls, even of things like os.system (pickle.loads(b"cos\nsystem\n(S'echo ACE'\ntR.")). Security cannot reasonably be retrofitted to such a model. With the likes of Serde (and indeed structural serialisers/deserialisers in practically every ahead-of-time-compiled language), this isn’t an issue: only the code controls things, and only types that declare a way of deserialising can be deserialised to.

Why can't we just save Rust data-structures in a memory-mapped file? That would make saving and loading much more efficient, and it would also make programming much more convenient.

I would encourage you to try and make that idea happen.

I mean, you can, if there are no pointers. But then you still need to verify the layout when you reload it etc.

Once you solve the various edge cases, this is basically what flatbuffers and capnproto are

The information what datastructures are in memory is based on compile time static analysis data, you wouldn't know what you're loading without some additional data annotations. It would also require unreasonably fixing memory layout implementation details between versions.

But these are things Rust can solve for the programmer! It can add annotations to the data and even provide versioning.

You can do this, but only if you know exactly what that state means. In practice this is almost certainly too fragile to be useful. If you change anything you probably invalidate your understanding of this data and cause chaos.

Rust's in memory data structures don't have any meaning on their own, they only have meaning in the context of the running program.

This is, in fact, the whole point of the New Type Idiom. If I have two types Minutes(u32) and Years(u32) those are represented in memory the same way, an unsigned 32-bit integer, just four bytes. But because Rust is statically Type Checked, everywhere the programs knows which is which, you can't assign a Minutes to a Years, or a Years to a Minutes, for the same reason you can't assign "Hacker News" to an f64.

But this means a 32-bit value in your hypothetical file could be either a Minutes or a Years, and a program loading that file needs to know a priori which it is. Once you start coming up with rules about how you'd do that, you're inventing a serialisation format, and you're exactly back to the problem where your original question arose.

> You can do this, but only if you know exactly what that state means. In practice this is almost certainly too fragile to be useful. If you change anything you probably invalidate your understanding of this data and cause chaos.

I've used the crate `savefile` for this in the past (https://docs.rs/savefile/latest/savefile/), which also supports migrating data between versions. So if you have a bunch of on-disk data of a struct and you add a new property, you can handle migrating the data on the fly to the new struct structure.

I'm also using `savefile` for saving HN items (comments and stories) to disk in my own HN application: https://codeberg.org/ditzes/ditzes/src/branch/master/src-tau... and it's very quick to load stuff from disk, haven't hit any bottlenecks really

Example: https://ditzes.com/item/35640002 (SpaceX Starship rocket explodes minutes after launch from Texas) has 1223 comments which are individually loaded from disk, takes around 100ms to load from disk, which considering how inefficient and unoptimized the code is, I'm pretty happy with.

> Once you start coming up with rules about how you'd do that, you're inventing a serialisation format, and you're exactly back to the problem where your original question arose.

Which is exactly what savefile does.

Sorry, but I think you are turning small hurdles into roadblocks in your mind.

It is certainly possible to store data structures directly inside files (I think MS Word did this to save documents), but indeed you have to think carefully about type safety but also data alignment, offsetting and such, and perhaps cryptographic signatures to make it secure when needed, which is what library/language designers need to think about to make it easier for programmers to use this stuff. Sure, it won't be adequate in all situations, but that's not what I'm asking for.

Besides the issues that others have mentioned (typing, storing nested data-structures, pointers), you would add a requirement on the backing OS and file system to enforce Mandatory File Locking.

While Windows enforces it, most Unix systems don't support it at all. Back when Linux had it (removed in 2021), it had to be enabled when a volume was mounted, and that was not default nor was it supported for all file system types.

Without it, even if you mmap() the file with the MAP_PRIVATE flag, portions of the file could be overwritten by other programs before you read them.

People like to send data around between different computers. Different computers have different endianness and potentially different padding requirements and differently sized 'usize's.

And there are the challenges with pointers and data structures others have mentioned. And supporting old files in the event of a Rust ABI change becomes incredibly hard.

We (the institution of programmers broadly speaking) tried that, and it was a hot mess. It's fine for scratch for a single machine, running a single version of the software. As soon as you involve multiple computers talking or multiple software versions over time, it quickly causes tons of issues.

There's a reason most serialization formats are either self-describing, have external schemas (json), or offload state to a specific state store engine.

It's a shame you're being downvoted, it's a perfectly valid question.

It is possible to do explicitly with `#[repr(C)]` in Rust. This is still fragile:

- `usize` and `isize` are not guaranteed to be portable between machines.

- pointers (including vtables in trait objects) are not portable between processes

- structs and enums may change in source code

- file descriptors, mutexes/futexes and any kind of external resources in general are not portable between processes. Thread-local variables are not even portable between threads.

In short: memory layout in general depends on a large amount of ephemeral compiler metadata. It is possible to constrain memory layout to something portable (protobufs, flatbuffers, capnproto, thrift, etc), but this seems more like a serialization format, not an efficient memory layout for a compiled language.

I'd say this is a good thing to avoid ossification of memory layouts: if you need to unmarshal something, it is better to mark this explicitly. Although the C ABI can be convenient, it constrains optimizations that compilers are allowed to do.

Honest truth, this is only going to side projects I wouldn’t even think of bringing this to production, it’s cool tho

Haha, quickstart requires toml.

materials: { "metal": …

are the second quotes optional?

They are not. Materials is an identifier, the name of a struct field, while "metal" is a string literal, the key of a dictionary.

Looks like something I'd like to crunch with Clojure.

There is not enough hatred in the world for the people who keep proliferating these.

Reply on news.ycombinator.com