Hacker News Reader

Comment by thesuperbigfrog | Hacker News Reader

>> This is a lot like schemaless databases. The flexibility of not having to fully specify the full schema does not mean that you don't benefit from specifying parts of it? Indeed, if you are indexing things, you have to specify those parts. But it is hard to argue with a straight face that schemaless tools don't have some strong benefits.

This is very similar to what Rich Hickey argued in "Simplicity Matters": https://youtu.be/rI8tNMsozo0?si=xTkpsLYTYh0jA5lB

Basically when you use a hash / JavaScript-style object rather than a fixed struct definition for your data (which amounts to a schema- less database in aggregate)

    Easy addition of new properties

    Co-existence of different versions of objects (newer objects will typically have more properties)

    Introspection of properties and validation rules / reflection

View on HN

The first two properties can be handled by adding optional fields to your struct. The third can be done at compile time rather than runtime if you define a strict class hierarchy. I think really the problem is that many languages make struct definition needlessly verbose so it feels convenient to just use a map. The price of using schemaless includes runtime performance as well as the compiler being unable to warn you about easy (for a compiler) to detect and report data mismatch errors. I have spent a lot of time using lua the way I think many people use clojure. It is quite seductive and does feel very productive at first, but eventually I find that for a sufficiently complex program, I want the compiler to validate my data usage for me.

Protobufs are an interesting hybrid. They assume a common, linear history of schema versions, where all data generators had access to an arbitrary version in that history. The schemas define field ids and types, but not which combinations of fields can show up.

If you can upgrade all the data then you don’t need to worry about versioning,

Or you can do it like HTTP headers and hope for the best.

Protobufs are also a fun place to look for how much debate people will get into regarding required versus optional fields. Your post is taking an implicit "everything is optional" view. But, it does allow you to be stricter.

Common Lisp Object System also touched on all of these ideas years ago.

Everything has been `optional` for about a decade.

> The option to set a field to required is absent in proto3 and strongly discouraged in proto2.

https://protobuf.dev/overview/#syntax

Ah, I dropped off protobuf a while back. I definitely remember a lot of uproar about it, at the time.

Sometimes I’ve wondered if versioned types might be a help in bringing coexistence to a typed setup, and if this is part of where the benefits of a service oriented architecture come from. But given how versioning snarls can play out in dependency management I expect this idea has some tradeoffs in the best case.

I'm curious what you have in mind that doesn't boil down to "duck typing?"

Not the most elegant mechanism, but it should be accessible to imagine an implementation via inheritance hierarchies even for a static manifestly typed language. Class "Person4.1" inherits from class "Person4" inherits from class "Person2" inherits from "Person". Probably there's a better way (and one could argue that a design where knowing the version of the class/object matters isn't a well-encapsulated design, which could have something to do with why duck typing often works out better than expected in OO systems).

I think my question is what would you be doing here that doesn't boil down to "has these properties/methods, I'll accept"?

I think this is a lot easier if you exclude the "methods" in my quote there. Since those almost certainly have the same general contract that would spread across things.

You might be interested in the Unison language then.

Reply on news.ycombinator.com