Hacker News Reader

Comment by wotbrew | Hacker News Reader

Mainly it is because the tar pit paper proposed the relational algebra in its functional relational programming.

I do think for some it might be easier to reason mechanically about dataflow through collection-oriented operators. But I suppose it's subjective.

clj-3df [1] to my understanding does something like Relic for datomic datalog using differential dataflow.

Disclaimer: I now work on XTDB [2], so datalog is somewhat now my day job.

[1] https://github.com/sixthnormal/clj-3df

[2] https://xtdb.com/

View on HN

Interesting. What are you thoughts in regards to Hickey's comments about the structural rigidity of relations/tables to represent information on how they impede flexibility, make your program hard to change over time and increase complexity.

He mentions it in various videos but these snippets are two quick finds:

- https://youtu.be/Pz_NvY1kw6I?t=489

- https://youtu.be/thpzXjmYyGk?t=255

I'm not the OP, but thinking well beyond the original topic of in-memory reactive programming (where my answer would be quite different) to the world of long-lived durable databases... one perspective to consider is that any system built around N-ary relations can automatically benefit from the full range of relational algebra for transforming and composing both base data and derived relations. The flexibility of N-ary relations is largely what has kept SQL databases relevant despite the flaws of SQL itself.

In contrast, a system that only handles base data in terms of triples assumes that you have a perfect attribute-oriented information model figured out upfront. But given this is rarely the case users will want tools that help them to easily transform/migrate their data and schema over time. Ideally this takes the form of a declarative language that minimises the amount of code that needs to be written. However, without a compelling end-to-end transformation language figured out I think any alternative database systems with their alternative information models (triple-based or otherwise) will struggle to compare favourably with mainstream databases, where declarative data munging with SQL is considered valuable and routine.

Triples may well prove to be the best way to handle information in software over the long-term, but I'm not sure that the systems which currently work with triples are good enough or widespread enough to test that theory.

Curious, where do you think datomic and datalog fall short (assuming one is using clojure and datomic's performance is tolerable)? By "perfect attribute-oriented model" I assume you are referring to the database's information model and support for that model, not to domain modeling with triples. Triples and relations are equally flexible but which models require more effort to maintain and transform along side with your application?

I think Hickey's point is that the structurally rigid nature of relations and stuff like arbitrary join tables needed to created many-to-many relations is that they get hard-coded throughout your application making your applications very hard to change over time, forcing you to put extreme effort to provide a set of logical views to isolate the application from the physical structural decisions. He says 'the more structural components(tables/intersection tables and having to name them(places)) you have in your model the more rigidity you get in your applications, but as you mentioned, triples haven't prove themselves.

I'm not experienced enough to understand all the tradeoffs here and can't tell if Hickey is wearing a salesman hat ;)

Of course, databases like postgres would be my first choice for most projects (even if using clojure which I like but is still a hard sell for webapps), as you said, they have too much going for them (flexibility of relational algebra, SQL, they are well understood, widespread, great ahd well supported implementations, etc..)

To be clear I'm not experienced enough to judge all these tradeoffs properly either, and I've never worked with Datomic. I suppose my main point is that an ideal database UX would avoid having to write processing/transformation code that needs to run outside the database (Clojure is pretty great, but still). This vision probably runs counter to the 'deconstructed' design of Datomic which heavily emphasises the usage of Clojure around it, but that could possibly be reconciled. It may well even barely register as a significant gap for current users in practice - I have no real idea :)

> provide a set of logical views to isolate the application from the physical structural decisions

To me this feels like the holy grail for what people really want from databases (beyond the basics of transactions/durability), and triples definitely still hold promise as a helpful abstraction. However I suspect Incremental View Maintenance (as Relic discusses) has a potentially even bigger role to play here.

I think this is a deep battle-of-the-approaches topic - whether it's better to "rewrite history" or deal with the different shape of past data. But in the history preserving case the data model doesn't need to be perfect up front, you can approach the data in your application in a way that takes into account the evolution. Eg in Datomic there are practices that support this [1] [2].

Of course you can have dev time scratch environments to play with stuff without to support

[1] https://docs.datomic.com/on-prem/best-practices.html?search=...

[2] https://docs.datomic.com/on-prem/best-practices.html?search=...

Reply on news.ycombinator.com