Hacker News Reader

Comment by jiggawatts | Hacker News Reader

After decades of experience, I'm starting to acquire a notion that most of modern web app development is simply the obstinate refusal to put the code where it really belongs: inside the database engine.

Impedance mismatch, ORM, type generators, query parameterisation, async, etc... all stem from treating data as this "external" thing instead of the beating heart of the application.

It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases.

Oh, the performance will be atrocious and grey-bearded wise old men will waggle their fingers in warning, but nobody will care. It'll be simple, consistent, integrated, and productive.

View on HN

That's....certainly a take.

it hurts that it's not exactly wrong.

but i don't think it's 100% right either, there are some things that you just can't do reliably, in current db engines at least.

As soon as you start baking this kind of support in to the db all you have if a db engine that has all the other bits stuffed in it.

They'll still have most of the issues you describe, it'll just be all in the "db layer" of the engine.

Yes inside the DB where it cannot be debugged or optimized

If you're putting advanced feature support into a db engine , you're probably also putting in semi-competent debugging support (at least i'd hope so).

But again, at that point you're really just moving the surface rather than addressing the issues.

I'm not thinking of "current" DB engines, but an entirely new stack that is end-to-end JavaScript (or WASM).

Something like Java's Akka or .NET Orleans combined with React.

So the "data" would be persisted by stateful Actors, which can run either on the server or in the browser, using the exact same code. Actors running in the browser can persist to localStorage and on the server the actors can persist to blob storage or whatever.

Actors have unique addresses that can be used to activate them. In this system these become standard HTTP URIs so that there is a uniform calling convention.

It sounds interesting, but its still just moving the problem surface elsewhere.

Bringing model definition and usage closer to the storage layer thereby reducing the need for translation and transport might cut down on the need for repeated and variant definitions across layers but it doesn't remove the other issues related to data storage.

There will need to be a system for storage and that will have to deal with transactional state management as well as consistency schemes, having that in an actor layer that is shared with other parts of the system might solve some issues, but it'd need to be really carefully managed so as not to inflict the solutions to those problems on the other parts of the system using the same transport mechanisms.

i dislike javascript with a passion but I'd be down for using a wasm based system that does what you say, my skepticism is usually just me shouting at clouds so it'd be interesting to see a working model.

You're 100% correct, but unless I'm missing something, this has already been done (modulo JavaScript, thankfully): PostgREST [0].

[0]: https://docs.postgrest.org/en/v13/

Let me introduce to you Convex.

https://www.convex.dev/

Disclaimer: I am the developer of SpacetimeDB which is spiritually similar. We absolutely intend to run client side as well. We need to for client side prediction. And eventually we’ll probably do web rendering at some point as well.

Did you, by chance, find here because I posted a link to spacetimedb? :)

I disagree.

First and foremost, what if there is not "the" database? What if you have multiple places that store data? For example, a postgres for ACID stuff, something like Kafka/RabbitMQ or similar to easily communicate with other services (or even yourself) - and sure, you could do that in postgres, but it's a trade-off. Then maybe something like redis/memcache for quick lookups/caching, then maybe elasticsearch for indexed search queries, and so on. And you usually also have some http API.

Sure you can say "I just do all with postgres" and honestly, that's often a good choice.

But it shows that it's not where "code (...) really belongs". Even IF you move a lot of logic into your database engine (and you often should), you most of the time will have another API and there will be a connection. Well, unless you use shared database tables with another application for communication.

All you do is pushing it out further to a later point - and often forcefully so.

> It terrifies me to say this, but sooner or later someone is going to cook up a JavaScript database engine that also has web capability, along with a native client-side cache component... and then it'll be curtains for traditional databases.

Not going to happen. Services like https://spacetimedb.com exist. Also, solutions like Spark (where you send your code to the database(s)) exist. And for certain things, they are great. However, it is a trade-off. There is no one-fits-all solution.

> Kafka / RabbitMQ

These are different things, and the fact that they're so often conflated is IMO prima facie that they're misused. If you need a queue, you shouldn't be reaching for Kafka, and vice-versa.

> Then maybe something like redis/memcache for quick lookups/caching

With proper RDBMS schema, indexing, and queries, these are _very_ frequently not needed. At FAANG scale, sure, but I think most would be shocked at how performant a properly-tuned RBDMS can be. That's of course the problem; RDBMS are incredibly difficult to run optimally at scale, and so the easier solution is to scale them up and slap a cache in front of them.

> elasticsearch for indexed search queries

Again, probably not needed for most. Both Postgres, MySQL, and SQLite (and I assume others) all have FTS that works quite well. ES is massively complicated to administer.

What you are saying isn't all that wrong. But it's besides the point that I'm making. Because it's not required to use ALL (or even more than one) of the tools or things (such as an http API) that I listed.

Reply on news.ycombinator.com