Hacker News Reader

Comment by refset | Hacker News Reader

> In databases like XTDB, the order of :where clauses doesn't matter since the engine is able to optimize them

Rich's observation was that query optimizers often get things wrong in ways that are hard to predict or control, but he's not fundamentally opposed to their use. That said, building a decent optimizer is a huge undertaking and I think they took the right decision to not attempt to bundle that sort of complexity into the original vision for Datomic otherwise they might never have shipped.

The state-of-the-art commercial engine for query optimization and execution algorithms in this 'triple' space is probably https://relational.ai/

View on HN

As I see it, Rich's stance is that of an expert in the database that doesn't need to deliver business features using the database. New users are not experts and even experienced users that work for companies have pressure to deliver features. You can get initial popularity by targeting these types of expert users working on more experimental products. However long term growth and popularity requires targeting the other 99.9% of users. I've seen one company adopt Datomic due to this type of user and then a couple years latter rip it out because as it grew it's developers were no longer of this type.

The original RDBMS vision was very explicitly for the users (both developers and analysts) to not have to be experts in their own database in order to achieve useful work, and without needing to think about procedural/3GL code from the get-go. In the intervening years query optimization has gotten a lot better, and hardware shifts have only worked in favour of this vision, but there's still a lot of work to be done before databases are truly "self-driving": https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-dr...

Until that's the case I can understand why people are tempted to bypass this traditional RDBMS wisdom, especially if they have a very strong conception about their data models, access patterns, and need for scale (e.g. see also Red Planet Labs 'Rama').

If you don't have RDBMS wisdom and program an application around a RDBMS, your application is going to suck.

No ORM or framework is going to save you.

In my experience, structuring a query to execute efficiency requires some basic software-engineering thinking about what's going on, while convincing a query planner to do the right thing requires deep expertise in the query planner.

> convincing a query planner to do the right thing requires deep expertise in the query planner

The advantage is that that can be somebody else's job though, and ideally (eventually) an AI's job.

Reply on news.ycombinator.com