Hacker News Reader

Comment by xwowsersx | Hacker News Reader

I've noticed a large number of stories on HN related to SQLite over the past few weeks. Maybe it's just random or I'm only now just noticing it, but is there some renewed/newfound interest in SQLite lately? If so, what's behind that?

View on HN

SQLite has been improving quite significantly over the years and continually impresses people in terms of performance, features and robustness.

When I first heard about it 10y ago, my boss (web shop) said it "isn't a real database", and that notion prevails in many circles. However more and more web developers recognize its benefits for production use. There are many use cases for it.

Most web projects should default to it IMO, because of its low operational costs and great performance, both in terms speed and reliability. It's incredibly straight forward to use, set up, back up and so on.

As an example: Wordpress, could absolutely default to SQLite as the vast majority of installations run on a single, shared host (apache/litespeed/nginx) and the data model is very simple and almost throughout stringly typed anyways. Running MySQL just adds friction and overhead for these use cases.

I think so. I attribute it to dqlite, rqlite and litestream that are making sqlite viable server side. It's always been viable server-side in tandem with a paxos/raft changestream (etcd streaming the changelog to an sqlite database.) but with the new projects providing out-of-the-box replication solutions it's even more-so viable.

It brings me so much joy to see sqlite get the love it deserves -- it's by far the best library I've ever used.

I think some people are realizing that SQLite has everything they need in their database use case, and they don't need a heavier, more complicated client-server database with more features.

This is exactly the reason. Some of us went off the beaten path and proved that you can actually put many users on a carefully-tuned SQLite instance (despite their official docs at the time). In many cases, you are able to exceed the capabilities of a one-node hosted DB solution due to latency reduction of in-process happiness.

Thanks. I thought that may have been the case, but I wasn't sure if there were also some recent improvements or developments.

I find topics tend to surge and wane, and sqlite is a popular one.

They don't always surge for a reason other than "someone posted something new about it and sent people down rabbit holes"

It was a meme in the "webscale" era that SQLite couldn't scale to many users, and was effectively something to use in desktop and client-side applications like mobile apps.

There have been times I've been talked down to in the past for using SQLite, so I just learned to shut up about it around ideologues. Now the tide is turning when it comes to webscale assumptions, and some of those ideologues' ideas have been proven wrong in many aspects.

SQLite performs well in read-heavy loads, even in web apps with many users, so it can fit those use cases well. I've been using it in a reverse search engine that gets a lot of traffic for going on 8 years now.

“Sqlite doesn’t scale”

https://twitter.com/levelsio/status/1520357256373874688

Dude gets 172 million requests a month.

To be fair, during that time the NoSQL crowd was saying that about all relational databases.

Yeah, at the time SQLite existed both as an archetypal example of "relational databases bad" and a case of "especially this one that isn't even a 'real' RDBMS".

I think it's this one (for context, the submission title was "Fly.io buys Litestream" before changing to the blog post title; you can still find it indexed that way in search engines): https://news.ycombinator.com/item?id=31318708

> ... is there some renewed/newfound interest in SQLite lately? If so, what's behind that?

Though i share that observation, in this particular case it's very likely coincidence. i've worked with Richard since 2008 on his Fossil SCM project, so have been "in that circle" for a long time without having ever actually contributed directly to sqlite. About two weeks ago the topic of wasm came up in a dev chat and it sounded to me like something interesting to experiment with (me being Fossil's "JavaScript Guy"), so i ran with it.

SQLite is quite famous for new developers as the default db in Rails and Django. There's a widespread misconception that SQLite is a toy DB and that it's necessary to switch to a real DB like Postgres or MySQL.

I suspect this fuels some of the hype--trying to correct the misconception.

well, let's be honest here: some of this is legitimate due to the nature of those frameworks, not necessarily sqlite. If you deploy a rails app with 8 process workers powered by a sqlite db even on a single host you're going to have database locking problems.

You have to be able to architect your application to send all write traffic to a single process and have a language that can keep up with your database while doing that. There are "better" architectures for this stuff now that will enable that which have come together in the last few years, and a resurgence in smaller indie projects willing to deploy sqlite on servers and do weird cool things like this fiddle or datasette

> You have to be able to architect your application to send all write traffic to a single process ...

Counter-point: sqlite's sister project and SCM, the Fossil SCM, is a self-hosting sqlite client application and has been since 2007. Every hit on the fossil-scm.org website is its own standalone fossil process accessing the same copy of the same db file, all while the developers are actively pushing and pulling changes to/from that same db and while half a dozen or so folks are logged in to its /chat room (each instance of which is long-polling that same db 24/7).

In 14 years of using that db i've encountered _maybe_ two locking errors.

Similarly, sqlite3's own forum is a fossil instance hosting a single sqlite3 repository.

That all writes "have" to be channeled through a single manager is demonstrably not the case.

Does fossil or the SQLite forum use rails or Django?

No. The SQLite Forum is just an instance of Fossil. Fossil is written in C.

I think this may also have to do with the indie community - SQLite seems to literally fit the bill much better than hosted DBs. A lot of what "the biggest companies" do doesn't go anywhere close to where most small (or even medium) teams' needs are. A lot of that additional complexity is simply not worth the extra effort and manpower (and oftentimes creates it!). "Lean programming" has become cool again :-)

Yeah definitely renewed interest. I think there's been a collective realisation that it's pretty good, and it has received some notable new features recently like JSON queries and proper type checking.

Probably a reaction to the ever growing complexity of software systems.

Lots of new stuff happening lately, yep! Mostly around distributed systems?

I've been keeping track of a few things here: https://gist.github.com/fideloper/ac9b81cee85003a59c8ad1a591...

Reply on news.ycombinator.com