Hacker News Reader

Comment by childintime | Hacker News Reader

Is it time to kill the database and replace it with a LLM optimized compiled version that simply implements the required API directly in (Rust) code, without any dynamic overhead? It probably will still be based of off a base design or a base file format.

Ultimately this system will encompass the whole OS, of course, but the DB might be the best place to start.

View on HN

You mean get rid of Postgres and build bespoke database-esque systems for every use case?

If so, then no. It is not time for that.

Yeah, I'm struggling to come up with a really good time for that.

The best I got is if you are trying to do an old-school style video game asset/save game storage. But even then, the value in just using sqlite or even parquet is really high.

There's so many really good data formats that deciding on a new one at this point seems pretty silly. Particularly because what you sign up for when you make a new one is losing any and all tools that could be used to work with and diagnose that data.

Instead of a database, the LLM will expose an api endpoint and build a database on demand?

That's interesting. Maybe to decrease latency the LLM could "cache" it's build of it's database and reuse in between instances. It could host this artifact on a "hub" of git trees and then any new use cases that come up, can be added to this git tree. Then it can possibly be reused in different use cases.

Yes you got it. Get rid of all the abstractions we've gotten so used to and effectively approach this as an embedded project, where every line of code has to be justified.

It tends to make sure you understand the system fully, as no foreign concepts need to be imported and deferred to. That should make your organization run better.

Just like Rust does, btw.

Thx.

Is it time to get rid of hammers and replace them with swiss army knives?

We could build a new hammer for each nail!

Why build one hammer and use it forever when you could pay $100 a month to build a new hammer every time you need to hit a nail?

for the database it is actually the opposite, as it is a swiss army knife.

look at the sibling comment, people are already doing this. with time only more people will.

an era is ending. there is a time and a season for everything.

Databases, like filesystems, are one of those pieces of infrastructure you do not want to build yourself unless you have a really good reason. They're complex, and you will not catch all the bugs yourself.

SQLite already exists and some people use it

SQLite is sure to provide a good parts toolbox. one may even use it as is ;-

I have seen this happening already at two different companies. And I'm also doing it as well. Particularly for search indexes where there's no risk of data loss.

Thanks for sharing.

Data loss is the obvious concern. Are you perhaps reusing parts of sqlite or postgres?

There'd be no benefit. They're good general purpose databases but for any specific workload you can do significantly better on your own. For search in particular even if you use Postgres you wouldn't want to use Postgres FTS you'd want to at least use Tiger Data's extension.

Unless you're tigerbeetle and want to handroll every single thing you do lol

everything code is going to be liquefied, maybe even including tigerbeetle. simpler systems are simply more reliable, and the effects compound (= the best part is no part). organizations will get rid of database professionals and buy their migration expertise externally, when they need it. it won't be from Oracle.

Reply on news.ycombinator.com