We're Building Postgres in Rust. Using the LLVM of Databases | Hacker News Reader

We're Building Postgres in Rust. Using the LLVM of Databases

turso.tech

109 pointspolyrand26 comments

Author also discussing it on their Twitter [0]. I’m reserving judgement for now, but this has lots of potential usecases even apart from replacing postgres.

[0]: https://x.com/glcst/status/2077759127682486561?s=46

People said same thing about sqlite, but I love turso sqlite and haven't faced any issues.

There is no turso sqlite

I don't know if you were being pedantic or confused, but turso is a sqlite3 remake in rust (with some additional features).

turso is not an sqlite remake.

at all

Maybe you could explain why you think it is not a remake, when it clearly is generally seen as one.

They are even going to the extend of preserving SQLite footguns, so I am not sure what else would it be.

It's something else that claims some level of compatibility with SQLite.

It has nothing to do with SQLite.

"An in-process SQL database, compatible with SQLite, and now with a Postgres frontend too "

I'm already quite excited about Turso being SQLite-compatible, but adding many features on top.

And when a feature is not directly compatible with SQLite (ie: you can't directly read the file with `sqlite3`, it's straightforward to convert). This is great because you know you'll always be able to continue working with that database. Even if Turso stopped working, it's still a valid SQLite database.

A combination I would be excited about is:

- Full support for Postgres protocol/wire format (ie: Postgres, but in-process, backed by a single file). - Optional: Client/server architecture for further scaling and remote management using existing Postgres tooling - All backed by a SQLite-compatible file

They are already adding MVCC to SQLite anyway. So their effort seems doable, and I hope they succeed.

Supporting Postgres is a good goal but honestly the real challenge is extensions. Would supporting Postgres wire compatibility guarantee any Postgres extension would also work? This is one of the problems with Aurora - it’s all well and good until you need an extension that isn’t on the blessed list

> Would supporting Postgres wire compatibility guarantee any Postgres extension would also work?

It would not, for obvious reasons.

Care to elaborate? The reasons are not obvious

Wire protocol applies to external interaction with the db.

Internal ABI/API is used by extensions to directly interact with core subsystems and depends on internal models for things like storage.

It might be analogous to HTTP versus an nginx plugin.

curious how hard it is to implement wire protocol compatibility vs the internal api surface. Are we talking like, an order of magnitude more work here? I wonder if, for example, Aurora team are literally maintaining forks of extensions for their implementation and that's why only a subset of all possible extensions are supported?

There’s a section about extensions in the FAQ.

Sounds cool but also why? Is it so it's possible to have 1 database and then switch from sqlite to postgres without data migration?

The llvm of databases is not turso, it's datafusion.

Yes I assumed datafusion when I heard that phrase.

datafusion is llvmm for etl engines, they don't have indexes or transactions for example.

> If you have ever babysat a REFRESH MATERIALIZED VIEW cron job, or faked live views with a pile of triggers, read that sentence again: our views update themselves, live. This is the feature Postgres users have quietly wanted for twenty years.

Real-time materialized views - so far mostly Chinese tech shops have solved this - look at Apache Doris, StarRocks - but these are built on top of MySQL then maybe RisingWave which is Postgres wire compatible.

having real-time materialized views means you can take out 1 or 2 infra pieces from your stack e.g Flink & Kafka - for some simplified use cases. that means no more ETL jobs for some use cases.

Would be interested in strategic value add versus cedardb (although cedar is closed source in a way that at the moment I think may kill it)

As someone who doesn't follow databases that closely, how is this different from Amazon Aurora (other than being open source)?

Tried DOOM in that page, it's really slow...

The ram is a blob on the db, ofc it's slow

Will it have fast COUNT-s?

Reply on news.ycombinator.com