SQLite is all you need for durable workflows
obeli.sk
obeli.sk
If you find yourself wanting things like an easy way to then introspect your SQLite database, figure out what is happening in the workflow, compose individual tasks, make workflows trivially callable, etc, give Temporal a look.
Alongside this, I have mostly moved away from files for agents. Markdown and JSON are great, but also feel like traps when building out smaller local apps. LLMs are great at SQLite and you can render anything you want out of it (Markdown, JSON, etc). It saves a lot of tokens when an agent can just query a specific row instead of having to fire up jq or grep through markdown. You get a nice portable self contained data management system that encourages agents to be more disciplined about how they structure their data than a bunch of files. It also continues to scale into MySQL/Postgres if your little local projects start to outgrow or become more formal, you already have schema and discipline around data.
On a modern processor, that's about GBs of data typically, right?
The underlying database isn't the most important thing. Just use SQL. Its namespacing (eg, through CTEs) is good and you're more likely to have colleagues who know SQL compared to jq.
As an occasional consumer of JSON/CSV, that's why I really like DuckDB, it's just SQL for such file formats. And it manages to be super fast at it too.
1. A robust durability implementation 2. A library of high performance data structure and algorithms
The fact this it's SQL is nice, but those two attributes are what make it great.
For example, I'm implement an in-process event log that I want to be durable. I started simple, but soon saw some edge cases and instead of playing whackamole I just swapped to using sqlite as an ordered kv store that gives me ACID.
Another example: ingesting multiple inter related datasets. Instead of a dozen hash maps in memory, I load them up into sqlite (no persistence) and then slice and dice as I need to.
It's a super useful tool.
Usually we end up writing a script to incrementally refresh a data-set I'm analyzing (or have someone send me a copy after they pull it).
I've been using sqlite for anything which needs an UPDATE - modifying a row deep inside the data-set with jsonl is a pain.
My github is full of java programs which update sqlite3 files with threadpools and a single big lock around the UPDATE (& then I write or have an agent write code to analyze it).
DuckDB is slowly replacing it in the context of python, simply because of the ease of pushing a UDF into the SQL.
Also because I really like expressing things as LEAD/LAG with a UDF on top.
Recommendations for good engines? Or just thoughts and analyses?
I wouldn't know, I've not done either, but I'd like to learn more from your or other's experience.
I think a genuine problem right now is people are building agentic work flows and learning the hard way highly reliable agentic work flows are hard. Agents are unreliable. They are both not deterministic and not the backing APIs have pretty high error rates. Temporal has solved that pain for me and made it easy to diagnose problems.
I don’t have anything really large scale running. But big enough that it takes billions of tokens and high reliability to finish.
ive been over here using claude relatively simply as of recent, just claude code and i might enter plan mode to do some bigger like scrap together a test suite of some sort, or i just have him scripting and refactoring/reformatting stuff under my direction. i wrote my own cli tool (needed to bake in the snowflake golang driver for external browser sso propagation) and added it as a skill so he can talk to our cloud dbms when im doing analytics things but for the most part its all pretty simple. feel like my productivity is 50x but after over a year with claude ive really backed off on asking him to do insane stuff and mostly keep him churning stuff out for me in domains i know very well.
so i read all this workflow stuff that needs durability and logging and im kind of astounded how many people have their AI stuff just running on their own round the clock. i didn't realize how much of peoples day to days needed to be automated, i don't seem to find myself surrounded by much that should be automated. jira is probably the only thing i need to sit down and automate because its such a translation tax on developers just so business people can feel involved. but outside of that... guess im behind the times, but i dont know if its that. i see the big grand things people use llms for ("im creating the ultimate knowledge base" or "ive automated everything under the sun and im making 10k a week" etc) and i am feeling either too tired, not ambitious enough, or unenthused by the creative and grand ways people are working with AI. seems like everyone has their own "perfect way to use AI" but I can't seem to find the oomph to go beyond using claude as a utility anymore. a year ago (maybe more cant remember anymore its all a blur) with claude in the sonnet era i was so amazed the first thing i did was try to reverse engineer a game using ghidra. had him building test suites to verify the math was correct. we were at this for weeks. my nearby datacenter probably drained 10 lakes. that was just one of _many_ over-ambitious projects i selected because of claude that never saw a finish line.
yesterday i opened beej.us and just started reading. im young and i feel like i somehow went from 'damn this claude shit is pretty cool' to 'AI is whatever its fine' in a year. like the bell curve meme.
I've tried clever tricks to get AI produce unsupervised stuff and came back from it. The slop and loss of cognitive knowledge about what it did was uncomfortable to me... I cannot understand how you would hand off critical job to it.
My devops coworker just shrugs, pumps out some yaml and helm and away it goes.
It really depends on your experience and tolerance for a lot of things.
Usually maintenance burden doesent start to make itself known till you get off the happy path or something breaks. Sometimes it can be a long while before that happens, sometimes it happens right away.
Other orgs have never heard of alerts or error reporting and naturally will not catch issues until they are catastrophic (for example services that crash frequently in the background go unnoticed until the crash frequency causes a catastrophic failure). In my experience a lot of issues are pretty simple such as running out of memory, CPU throttling, crashes caused by simple bugs (nil panics). If you have good observability you can catch those issues early.
For example: people rag on Ceph that their cluster somehow got into a broken state, but that really only occurs when abuse of the ceph cluster has went on long enough that the cluster finally reaches the tipping point where it is unrecoverable. If you set ceph up, follow the correct replication rules so components are spread across failure domains, and use the metrics and alerts that are distributed with ceph it is actually quite hard to break the cluster.
As best I can tell it doesn't do any batching of it's writes/reads, and it's update heavy in places rather than append (I suspect their cloud version might do some of these things)
It's pretty close to "let's make every function call serialise it's parameters/return value, go through a postgres table and several network hops"
That said it can be very useful, but it's a heavy tool that's best suited for high value/risk workflows where you're earning enough from the execution that you can afford the overhead (for example an Uber trip with several dollars of service fees is probably a good fit, unsurprisingly since it's roots are from Uber)
They ship Helm charts so reality is somewhere between "helm deploy" and "substantial ops burden". I don't have to touch it very frequently, but that is not to say I don't have to touch it. There's occasional releases and there have been times where (probably due to my inexperience with helm) I botched an upgrade and lost some data. And I've been on this journey for years; when I first started, they didn't have a Python SDK and it was one of my (many) excuses to learn Go. But anyway to your point, yes, if you're comfortable with k8s and Helm then you shouldn't have much of a problem running hundreds of thousands of workflows; if you want to really push the throughput and optimize cost you probably need to get creative the individual services and look into cassandra (maybe? idk).
Self-hosting is very easy in my experience, I've done it for 2 years but management wanted to move to Temporal Cloud. They have a helm chart which just works including upgrades. This does assume you have the whole k8s shebang set up and working in your company. I never had to touch is outside upgrades which took maybe 30m including validation.
We use it heavily and everyone who started on it doing simple scripting/automation all love it, everyone who built real production systems on top of it all hate it. Possibly operator error, but my experience hasn’t matched the rosy picture painted in these comments.
I also have restate.dev on my reseearch list, which on paper should scale well and be definitely more lightweight and simple to setup, worth having a look.
Determinism sucks, you do have to work hard and make everything idempotent in activities like we would for durable software anyway. The language we used was incorrect (Go) and has a lot of boilerplate compared to alternatives we later investigated (Python and TypeScript). Visibility can be slow and misses information. We needed to write our own APIs to work effectively with Agents for root-cause analysis of failures.
With all the caveats - Temporal is amazing, it feels much better than previous orchestrators I used like Prefect or Airflow. 100% would adopt again.
- Temporal itself is written in Go and we use Go for our backend so we expected this to be a natural fit. - Temporal makes writing activities in Go very explicit and boilerplatey - This in turn makes testing more difficult than it needs to be often - Temporal doesn't play well with Go's concurrency model at all (all stuff like goroutines needs to go through its special workflows.Go) a lot more often you have to write stuff that "appeases" temporal. - The whole workflows.ExecuteActivity(...).Get(...) is weird, having futures in a language explcitly designed to avoid that is weird. - All our compute isn't done on temporal workers anyway, its done (in another AWS account, owned by the customer) in batch compute (aws batch, lambda, ec2, whatever) so our temporal code isn't CPU heavy but is highly concurrent and needs a very high reliability guarantee. - Compare that to temporal with TypeScript, where it's simple and easy to use the same code inside or outside of temporal. Testing is trivial and the code looks like "regular code".
There are no free lunches in this space. I have no idea how good or bad Temporal is since my usage is pretty small and isolated, but software rarely just works and impresses me and Temporal for my local machine orchestrating genuinely did. I think Netflix's conductor is another cool option, but I ended up with Temporal due to license.
I see this as Temporal surfacing inherent complexity of the domain in a way that forces the developer to consider it, rather than introducing extra complexity.
If it didn't make workflow determinism a strict requirement, the requirement would still exist - it would just hurt much worse in production when it's broken.
See also: Rust borrowing
If you're fine with deploying several versions of workers (and are on a reasonably new version) you can just avoid the determinism issue altogether with their k8s controller.
If you do need to have some long workflows, there is an explicit hook for "what happens to existing workflows on version upgrade".
But to be fair - none of the other orchastrators I used (like AirFlow) made me write workflows.IsNewCode/IsOldCode like temporal does. On the other hand AirFlow doesn't even have the capability to do that in the first place (or at least it didn't last I used it).
That said, I appreciate this is hard in practice. We need to start small to manage the development rabbit hole risk, while also wanting to dream big. There is a tension there that I find hard to balance.
I rewrote the pipeline in Python (a correct version of it) with state management in SQLite and logs in plain old flat files, and everything has been running smoothly ever since. In fact this is the only data flow that has worked without errors or interruptions in the last six months.
Instead of replicating the db file with Litestream I do a remote backup with Restic before and after each run; it's not an exact replacement of Litestream as we could possibly lose a whole run if the machine died / disappeared at the end of a run, but it lets one restore any day very easily. In an ideal world I think we should have both (live replica + backups).
Just wanted to make sure no one missed this point in your comment because eventually users will be paying the full cost for tokens instead of VC's paying, with GitHub Copilot's pricing realignment leading the way.
This is a foundational principle of computer science. It seems to me that the "SQLite for everything" crowd is a little bit inexperienced.
For me the main benefit of sqlite is that it's a library rather than an app.
Im not saying that sqlite isn't useful, im mostly saying that using postgres doesnt have to be complicated.
I've been assured by many HN users that running apps/sites on a single VPS requires near-zero maintenance or monitoring to achieve acceptable uptime 24/7/365 for years on end, sooooo...just pretend it will never fail like your main server process?
There's a world between "local file" and "network DB server", running a DB server locally has lots of benefits from being able to easily query from outside if needed to forcing you to consider concurrency without the latency overhead of a network hop.
Some people want some of the benefits you get from SQLite.
SQLite is obviously not perfect, but it's an incredible piece of software, and people regularly find good ways to make use of an excellent pieces of software.
That being said I'd kill for someone who used it and benefited to explain it to me in a practical sense. (specifically where syncing is involved, and syncing a subset of the SQLite is necessary. If it's "just" a document store thats treated like a blob for syncing/backup, that's familiar. If it's all in one storage but only local, that's familiar.)
Re: TFA, I guess it would have helped if I knew what Obelisk was, which is on me, and a more in-depth explanation of how this ties into AI/agents, which is on the industry/writer.
I don't want to be glib and leave it there, even though I'm slightly annoyed you missed several sigils in my post that I was well past that.
The point is, for the not in your pocket case, for the not a singular document store case, I'm curious what the use case is.
SQLite is a nice local store. It's this server stuff that I don’t grok, well, yet. :)
Yes, in theory: given a large enough database, and a disk that can only do one operation at a time, and a large enough operation that touches enough of the database. In practice, in a SQLite single tenant scenario? No, not at all.
> what difference does it make if you write sequentially vs concurrently. Why does concurrency even matter for databases?
As soon as your codebase involves reacting to events independently of a user taking action it becomes a practical concern. Generally, this is a broad question and has 1,000,000 answers.
EDIT: Originally I had "I think you understand generally, no?" appended but realized that's not helpful at all, if you did, you wouldn't be asking.
Something that may help is imagining what'd happen if a DB wasn't thread safe / didn't allow multiple writers. Ex. in SQLite's case, it allows multiple write operations to take place but there's a one-at-a-time queue. If we didn't have databases that were able to execute multiple writes simultaneously, you'd need a separate database for each concurrent writer you expect, and you'd effectively have a global lock. Orderly scaling would be ~impossible unless you did something crazy like have a single server per user
Edit: thanks for clarifying in the edit, makes a lot more sense.
TL;DR: whatever works for you is the right decision. (which isn't helpful, I heard this so many times and as the recipient, I thought "That's nice. Now how do I choose what works for me?")
I finally had to use Postgres a couple years ago after a career of only SQLite - startup founder & iOS app developer using SQLite, turned Googler on Android, turned doing-my-own-thing.
In retrospect, I have made only one bad decision:
I went way out of my way to make SQLite work at my 2009-iOS-startup. It was a restaurant point of sale system, and to allow a networked system, one of the iOS devices would act as a server. This was a really cool trick, even an advantage in marketing that was appreciated by users. It meant the restaurant could continue to operate if the internet went down. But it eventually became clear owners loved having internet-based access too, ex. to do reporting/financial analysis over the data. And I kept contorting, instead of moving past my fear of getting into things I didn’t know, I instead did some like rudimentary thing over port forwarding. The bad decision here was riding one horse for so long and letting it affect the product, having a real server database would have allowed for a lot more features, think, first party gift cards, and a 100 others.
After leaving Google I needed server-side storage and fought and fought to avoid it. Then it turned out Postgres is easy and, just like SQLite, 99.999% of the time I don’t even know I’m using it.
In retrospect, there’s ~0 switching cost to these, particularly in age of LLMs. If you do need something more one day, it’ll be easy to do, and if you have to do it in a rush because you’re successful, you’re in Good Problem territory.
Hope that helped, after writing it out, dunno how convincing it is. Feel free to follow up, I appreciate the curiosity/framing because I had the same thought for so long.
Speed is rarely the constraint that makes it unsuitable for an application.
Like you suggest, the reason for not picking SQLite is not reliability, speed, etc. Networked DBs allow decoupling between app and db servers, which have operationally different characteristics. But most importantly, you can have multiple apps access the same DB at the same time. Eg analytics, one off queries, any 3p app that interacts with your data directly.
For a simplified example, having three processes reading blocks X, Y, Z in parallel is much faster than having a single process read block X, wait for the read to finish, read block Y, wait for the read to finish, read block Z and wait for the read to finish.
> "SQLite for everything" crowd is a little bit inexperienced.
every time i see it in a real application, it becomes a huge focus of issues (for example: jellyfin, hermes, openwebui, comfyui)
How exactly is this a foundational principle of computer science?
SQLite is an excellent production db for many real world workloads, as has been widely documented. It is very different to Postgres, so requires learning a whole new skill set.
One way to think about it is that SQLite can work well for the parts of your system where there is naturally strong partitioning.
Or the parts of your system that don't have big data and no need for massively concurrent writes. And that's the vast majority of systems!
If you think the majority of systems require massively concurrent writes, I think you need to look a bit harder. SQLite is, after all, the most widely deployed database system, ever.
It was forced upon most of us(not me, I used BeOS then Debian then FreeBSD).
I deployed phoenix.
It is not «a foundational principle of computer science».
[0] https://www.postgresql.org/docs/current/sql-createtable.html...
2. You should check if your backup method backs up unlogged tables. For example, RDS Snapshots on AWS do not backup unlogged tables.
These 2 are a double whammy where if you aren't aware of these tradeoffs you can find that a bad restart has deleted all your data, plus your unlogged tables were never backed up.
130k tps even with unlogged is not always super easy especially if getting hit concurrently. Postgresql connection overhead alone can be pretty brutal if you are setting up and tearing down connections or have 1,000 writers etc.
Postgresql generally requires good network connectivity. Folks doing sqlite distributed tend to have everything independent, you literally don't need to worry about connection / security / firewall / permissioning / internode escape or data leaking etc, can even have problems in local side networking and services can still serve.
SQLite is likely used more than all other database engines combined. Billions and billions of copies of SQLite exist in the wild. SQLite is found in:
Every Android device Every iPhone and iOS device Every Mac Every Windows 10/11 installation Every Firefox, Chrome, and Safari web browser Every instance of Skype Every instance of iTunes Every Dropbox client Every TurboTax and QuickBooks PHP and Python Most television sets and set-top cable boxes Most automotive multimedia systems Countless millions of other applications
And no, being a part of the python standard library doesn't means it is being used by the average python user. These days I'd say at least half of them are just there for machine learning.
I visited a piano store once that was running everything off MS Access. If only they had switched to HA technologies, they would be able to sell millions of pianos a day!
Sqlite is used in real production apps more than any other database.
Sqlite is also weak at any sort of write concurrency.
Both can be true.
You can have as many as you want - and one is often plenty.
Hipp even said that it is not a replacement for a real multi-user, concurrent RDMS. Its primary competitor is "fsync".
The only thing you really give up is HA/failover and DR. But there are solutions to deal with those. And single-server systems are generally surprisingly robust (since, in the absence of very complex control planes, uptime goes down with more systems).
Just use Postgres with ro replicas.
I've had pretty terrible experiences with SQLite and Longhorn/NFS.
It's just not the right database for pretty much ANY network based filesystem, where the locking primatives aren't as robust, and you might get two processes trying to hit it at the same time.
Frankly - they say this themselves: https://sqlite.org/howtocorrupt.html
As someone who runs a fairly big personal cluster backed by a mix of giant NFS storage for media, and relatively large longhorn SSD drives for configs/temp data...
I avoid sqlite backing like the plague. It will get corrupted. Period. It's not the db for this use-case, and I'll take postgres/maria/mysql/mongo/ANYTHING else over it.
If you do it - back it up ALL THE TIME, because it's going to get corrupted.
I'm a big fan of re-evaluating prior "best practices" in light of technology changes, especially in ways that improve simplicity. Running my family's social media site off a single sqlite DB on a VPS is great. ~15 users, almost zero maintenance. I run my FreshRSS instance off of sqlite, as well as my "now" page. At work, I used sqlite for all kinds of things over the past decades: as an ad hoc job queue, as a quick way to ingest and query lots of logs locally, and present/filter in realtime with simonw's excellent https://github.com/simonw/datasette.
I don't think it's every "sqlite for everything" as much as it is "sqlite in lots of places you probably didn't think to apply it."
kentonv/Cloudflare's work on sqlite at the edge might have made this thinking a bit more popular, but it was always around. https://blog.cloudflare.com/sqlite-in-durable-objects/
I suspect being aware of all those little neat cases and wanting to leverage sqlite for them may be an indicator of experience, rather than the opposite.
Details, please!
Unlimited-length posts in a chronological feed. You don't have to subscribe to everyone - having something appear in your feed is opt-in.
Circles. Subscribing to someone into a hobby can get noisy. Circles give the hobby a place, for those that want to check.
RSS everywhere: Anyone can add an RSS feed to the server, and anyone can view all the subscribed feeds and choose which to follow. They are not part of the home feed, but a separate section. Every feed (circle, user) has a public-to-the-internet feed and a private-to-folks-on-the-server feed.
Mentions: you can @-mention anyone on the server to get a post into their home feed.
Public posts: by default, every post is private. If a user checks a box, the post will be made public, so folks that are not logged in can see it.
Posts in the feeds section, circles section, or home section can be replied to, with the user choosing where to share (a circle, or their feed)
It's all a single-file Python script that can run in either CGI mode or server mode. I compile it with nuitka and run it as a CGI behind Apache. Very old school, works fine. Non-attachment data stored in sqlite, so if you have the DB, you can fire up a copy of the site, sans attachments.
Attachments: gallery posts supported, with lightbox viewing. PDFs supported. Nothing else right now.
Works on mobile.
No email or other form of notifications. If users visit, they see stuff. If they don't, they don't.
Super-opinionated: admin controls everything, and password resets go through the admin, who simply asks the server to regenerate a new password, that the admin then passes along to the user.
There are no direct messages, or private posts, in the sense that if you log in, you'll be able to see everything going on if you click through to it.
Replies, comments, and reactions are supported. Conversation view (tree of posts replying to each other) is supported.
If on the other hand they're talking about single-user, software in the small - hell yeah. In fact, I'd also promote DuckDB in this regard (mostly for analytics) - with the power of a single machine these days, you can do a surprising amount and never have to worry about distribution. Unless you know you'll have to, in which case you're probably just digging yourself a hole?
Second, I think you're overestimating how hard sharding is here. There are plenty of use cases for which sharding isn't just easy to set up, but the natural thing you'd be likely to do even without scale. Things like e.g. a helpdesk SaaS, where each customer/organization has it's own independent data.
Third, a large part of the point is that you are unlikely to know ahead of time you're going to "scale like that". As I already pointed out, most SaaS apps do not end up having many users. For some that's intentional, but for others the reason is that they simply never caught on. For those cases (and they're the vast majority), cosplaying as a much larger app is a complete waste. It's much better to wait until you're successful enough to need to switch and then use the revenue you now have to solve the scaling problem you ran into.
Fourth, as an aside, ironically the other (slightly less) easy way to shard superficially resembles a common way people cosplay as netflix: splitting your data by domain as "microservices" (although there's a good chance they don't need to be independent processes/on independent machines).
The concerns are more about distributed systems issues, maintenance, etc. Typical NIH issues. Which you're embracing when you start building your own systems (ie: with sharding).
This is a solved problem, why would I want to inject these burdens onto myself, instead of solving ... actual problems that I already have enough of! I don't want to be building another Temporal/Cadence/DBOS/etc.
I see obsessions with tooling/solutions constantly from experienced devs who fall in love with the original solution and think it's the only way to do things -- so the experience part cuts both ways.
Plus, D1 has a 10gb limit which is wild to call “sufficient”.
This is even more obvious when you start doing transactions processing an row locks across the network limit you to 1-3k TPS that you cannot scale out of (Pareto distribution is merciless).
[1] - https://andersmurphy.com/2025/12/02/100000-tps-over-a-billio...
In the real world we are looking at things like RPO (recovery point objective) and RTO (recovery time objective). You need to consider HA and DR. It’s in these areas where SQLite does not scale.
That’s why I struggle to see the fit for SQLite in any sort of multi-user server environment. If you need the data to be durable, then the bigger DB’s have the tools. If you don’t need the data to be durable, just keep it in memory. I’m sure there are niches I am missing.
You can definitely go faster over less data doing single inserts on a better stack, with weaker guarantees.
RPO litestream even in it's default settings gives you point in time streaming backups to the second, which is considerably better than what RDS five minutes. So the funny thing is the durability guarantees are worse with the "bigger DBs".
RTO again you can have a standby that's warm with a copy of the data through litestream. Regional sharding also becomes trivial.
It's a solid set up for a lot of products/apps. Postgres is still fine if you want things like roles and permissions etc. Or if you don't have experience getting the most out of sqlite.
It doesn't play nice with other things running with it in practice. JVM and postgres on the same box is a textbook bad time.
The point is to survive the Pareto row locking problem you need to move away from a network database (if you want to still have interactive transactions). The network part is the main point of a network database, once you drop that there's not much pointing sticking with the added complexity unless there's another feature you really need.
This makes scalability _much_ easier to reason about. It's cut-paste, cut-paste. Every N users needs another shard.
It does buy you a _different_ set of problems, like cross-shard querying (analytics) and how to do load leveling as users age out.
But it avoids the whole shared index scaling problems from inserts/updates with large user counts.
It becomes a hierarchical instead of a relational database.
you seem like the inexperienced one to me..
also, use-case for massively concurrent writes is pretty narrow, and SQLite is not optimizing for that anyway.
There is irony here
The combination of SQLite (libsql, a concurrent implementation of sqlite) and Rust means I can do so from a $2/m VPS and a single server instance.
Backups are done via a cron job that uploads to S3.
Does it pass the "Netflix scale" test? No
But it doesn't need to. I'm not profiting from the service and SQLite offers a path to scale if/when ready because... well it's just SQL and I can literally just swap `libsql::Connection` with `psql::Connection` in my repositories.
Plus upgrading from a $2/m VPS to a $10/month VPS quadripples the number of concurrent users I can support.
IMO, you can vertically scale extraordinary far with SQlite and an efficient server implementation.
I'd wager that 90% of forum websites, wordpress sites and online shops would be fine with SQLite.
You can probably do it with regular SQLite, too. Being limited to a single writer isn't as devastating as it sounds when they get processed very quickly. Probably don't need Rust either but it'll be more efficient than the usual choices.
(Also, it looks like libsql is the same as SQLite? Only Turso has concurrent writes)
I think people are afraid to read the documentation for postgres. You can start it up in milliseconds. Fast enough and light enough to run one copy for every test case in your test suite, or whatever you're using it for. (mkdir /tmp/whatever; initdb -D /tmp/whatever --no-instructions -A reject -c listen_addresses= --auth-local=trust --no-sync -c fsync=off -c unix_socket_directories=/tmp/whatever -U postgres --no-locale; postgres -D /tmp/whatever) Now you have a test database that behaves exactly like production because it's exactly like production. (OK, turning fsync off makes it a lot faster than production, so be careful.)
Postgres may introduce a single-file embedded filesystem because what the hell, but the irony is all these guys won't even notice it. The same people that say Postgres backups are too hard.
I think docker is still super underappreciated so setting up any kind of server is seen as a chore. In my eyes it makes running tons of services like this very easy, so ill take the extra functionality, extensibility etc of postgres.
Don't get me wrong, I've worked with plenty of server-based databases, including proper dedicated database servers. It's great tech and often the best tool for the job. But not always and I'd argue not in the majority of uses.
You seem to be talking about a vastly different use case.
Containerized apps having their own database? What? Aren’t these types of containers stateless? I always very much try to keep state out of app containers.
What kind of data storage are we talking about?
Those database containers get a PVC/volume/mount for their data dirs. The only thing ever connecting to them is their "owner" application container. So at that point, why not drop the postgres container and PVC mount a sqlite directory in the app container? The result is the same.
Honestly, this whole leys run loads of nodes seems to have sprung up from languages that are slow oe don't have decent concurrency.
Especially since in production you might want to scale the parts separately. I like to have a Postgres cluster to connect where backup is already handled, and the app then doesn’t have any persistent data, doesn’t need any network volume mounts.
2. K8S DaemonSet, PVC backed by probably Ceph
3. Just..don't care? Do maintenance outside of working hours, fix issues quickly and explain things nicely to your customers. Not everything is google-scale. Most people can deal with some downtime.
And it's not like you won't have downtime in let's say a postgres-backed app. But now you have two "servers" to deal with.
I guess I am just not used to downtime being acceptable. Spent most of my career working for a CDN, any sort of downtime was simply unacceptable. I can't stop myself from that sort of thinking now.
is just wrong, and I don't think that the SQLite fans are that crowd. Taking a database server for everything is probably possible, but often unnecessary. With experience, one can properly judge when SQLite is sufficient and when it is not.
So arguing that the SQLite crowd is inexperienced feels weird, because inexperienced people have a much harder time judging when to use what and can just use the database server all the time (even when it is overkill).
Yes, but that's not its main selling point. An SQLite database is also a single file, which makes it incredibly easy to replicate, backup, transfer, restore, etc.
Files which you cannot just copy while your application is running if you want a correct backup.
You say that being an embedded database isn't the main selling point, being contained within a single file is. But that's a completely normal feature of an embedded db, to the point that the one implies the other.
> SQLite does not compete with client/server databases. SQLite competes with fopen().
"Just use postgres" "Just use sqlite" "Juse use a monolith" "Just use sftp" "Just use an ec2 instance"
Usually these people have flunked out of the school of (distributed system) hard knocks. They couldn't hack it and are retreating to familiar.
The funny part is when one of those people fluke themselves into senior management when their saas takes off.
Inevitably they have to suck it up and hire experts in the same technologies that "no one needs".
Scalability = success. We need to be "scalable" because that means we're successful right? Scalability = real engineering. I'm a real engineer so I need to design everything to be "scalable" because I'm so smart
>The funny part is when one of those people fluke themselves into senior management when their saas takes off.
>Inevitably they have to suck it up and hire experts in the same technologies that "no one needs".
Sounds like they were the wise ones to build something simple that achieved a high level of success.
:page_size 4096
:journal_mode "WAL"
:synchronous "NORMAL"
:temp_store "MEMORY"
:busy_timeout 5000
(Synchronous FULL in context where it matters)
But, it depends on the shape of your data, your indexes, how much of the data you care about is filling up a page. If your distribution is more even you sometimes need more cache than a more Pareto distributed data set etc. Things like not caching prepared statements costs you more (you should almost always be caching prepared statements per connection with sqlite).
You have to give things more thought at a billion sure. Partial indexes are your friend. You'll also want more cache to prevent thrashing etc.
- https://andersmurphy.com/2025/12/02/100000-tps-over-a-billio...
- what is X, I just do Y
- wow I can see so many limits of Y, now I do X
- I use X for literally everything
- now that I properly understand the limits of Y but also the heavy constraints of X ... maybe Y is enough
- I use Y for literally everything
rinse & repeat. The thing is with actual usage and actual context one does learn and thus can get away with a lot more "basic" solution but it does require genuinely understanding the limits.
It is a source of expertise, because you really learn a lot from it. But when you become old (43 over here), you really learn to appreciate “boring” solutions.
So, xBase was all you needed in the mid 80s. Then DBM was all you needed in the mid 2000s. Now, in the mid 2020s, we're told that it's SQLite that is all you need.
It was partly true then, and partly true now. But the full story's always been more complicated, so it's still worth considering a range of potential solutions rather than relying on simplistic rules of thumb or slogans.
(Wake me when nosql comes back into fashion, I'll be able to do a great "old man yells at clouds" routine about that one...)
1. Intercom 2. Zendesk 3. Email marketing 4. Kanban 5. Todo 6. Our billing stack 7. Our issue tracker 8. Our forum 9. Uptime monitor 10. PagerDuty (clone)
I have dozens of products I sell, so I thought: why not build everything ourselves?
All of these run on the same server and use very little memory. I replaced all the SaaS tools we used with these.
I also moved to dedicated servers and dropped costs to about 1/10th of what we were paying for managed cloud solutions, while maintaining the same HA and even achieving lower latency (partly because noisy neighbors on VPSes were increasing tail latency).
We used to spend a ton on this stuff. These have now been in production for four months and have only needed minor updates.
Deployment is dirt simple. No Docker, no Kubernetes—just a systemd service and a binary built on the dev machine and deployed.
We also used to pay for services like MaxMind and IPData. I ended up hand-rolling my own IP geolocation service, which, in my tests, outperforms most existing solutions.
It all started with replacing Uptime Robot. Then I got more confident and replaced PagerDuty. After that, I replaced Intercom.
Finally, I had always heard people say, "Don't build your own billing stack." But I said YOLO, let me make that mistake myself. So I studied our existing billing solution, developed my own, and rolled it out. So far, we've had zero issues with it.
Caddy in front.
I found that we only use maybe 1–5% of the features most SaaS products offer, while the features we actually need keep getting buried deeper and deeper inside these "enterprise-grade" platforms, making our workflows more difficult.
I won't show my commercial products because our partners and clients probably wouldn't appreciate knowing how cheap I am—but I call it being resourceful.
I can show my free app, though, which has 20,000+ users and was launched recently: https://macrocodex.app/
It only uses the Zendesk clone. Email is handled through Cloudflare routing, so we pay almost nothing to run the app.
I just have uptime service hosted outside of our main infra. It connects to my service called Siren, which alerts me on my phone with an alarm on full volume with SWAT cat intro.
It's good enough for what we do, barely have any downtime. But it helped me figure out 6s downtime we would experience when our spot instances get knocked out, so it helped me increase health check frequency
6s downtime is a lot when you are getting hammered at 100 RPS.
Quick question: how do you inspect database contents of your deployed (micro)apps? I used SQLite in one production app and what I didn't like was to use terminal sqlite client on the server or having to copy DB to my laptop to query it with sqlitebrowser. With my Postgres server it is much simpler to just query by rich GUI client with SSH tunnel.
What happens when you get hit by a bus or someone with higher annual revenue tracks you down because of this comment and hires you away from this custom software stack with a bigger slice of the profits?
It sounds like there's also a bus factor for your server but I'm sure you're aware of that though it sounds like your clients aren't.
Besides keeping the costs down, I find that this approach makes it way more enjoyable to build and easier to manage than having 10 services/subscriptions.
*PS: I think that you could add a small QR Code for the iOS/Apple installation on your app's website.
1) How do you do backups? Do you use github.com/benbjohnson/litestream? CRON job backup with rsync?
2) Any issues with large databases and many clients? Is there a TPS or DB size where SQLite becomes problematic?
3) How do you deploy new binaries and safely shutdown the old instance? Caddy change to route to new binary + Go's HTTP server graceful-shutdown on old instance?
4) Do you use a pure-Go SQLite lib or one of the CGO libs?
Personally, lots of ETL can just be taken care of locally without involving enterprise databases. In such cases, DuckDB is 5x-10x better than SQLite and orders of magnitude simpler/faster than spinning up a dedicated Postgres database.
For general scripting, there's no match between a 20-lines awk script and a much cleaner, robust, maintainable equivalent SQL script based on DuckDB.
I just hope MotherDuck don't need to pump/dump for IPO - it would be sad losing that tool for the usual corporate greed.
Second, it's funny you should mention the 20-line awk script. I was making a very similar argument yesterday at the Ubuntu Summit: at some point, using shell scripts with GNU coreutilus becomes impractical, while DuckDB SQL scripts scale better in terms of complexity and maintainability (and often also performance). My slides are here: https://blobs.duckdb.org/slides/duckdb-ubuntu-summit-2026.pd... (pages 32 to 36)
Third, MotherDuck develops a closed-source DBaaS on DuckDB. They build on DuckDB, and you connect to MotherDuck with DuckDB but they are a separate VC-funded company headquartered in Seattle. DuckDB is developed by DuckLabs, a bootstrapped (revenue-funded) company in Amsterdam. And the IP of the project is in a third organization: a Dutch non-profit called the DuckDB Foundation. For details, see https://duckdb.org/faq#how-are-duckdb-the-duckdb-foundation-...
I've been testing different storage engines for my agent harness and I can get up to 7.5k concurrent sessions on a single vCPU with SQLite whereas Postgres crashes or runs out connections.
[0] https://github.com/impalasys/talon/pull/23#issuecomment-4577...
In the context of SQLite being understood to be a quite excellent piece of software - shouldn't we expect it to be?
In the context of a single-node, Postgres is overkill. It should not be expected to be competitive with SQLite.
This is almost like benchmarking an in-memory HashMap to Redis and being surprised that it performs well in ideal conditions.
> Versions before 1.0 do not promise syntax/API stability or storage compatibility.
Also thanks for the incidental exposure to a DB I'd never heard of before... with a browser-based demo CozoDB may be a good way to start experimenting with Datalog.
Leaving the current thread is where you lose the game in terms of latency. SQLite can work on timescales measured in microseconds if you don't force interthread communication.
Pedantically it's an in process virtual machine for operating on structured data. Which is precisely where it shows it's weakness, in my experience, when you end up with complicated table structures and complex join mechanics you then need to start thinking ahead of the query planner and VM code a bit in order to maintain reasonable performance.
There are more than a few unusual things worth knowing:
------
I see this kind of YAGNI thinking a lot, but in my view, it must be balanced against the effort you'd put into resolving any edge cases and adapting current architecture to your use case.
Imagine you deploy Sqlite, and thought it fine by itself, you keep running into some unforeseen challenges with the use to which you are putting. YOu'd need to sink valuable time and effort into addressing those. Then, when you have outgrown it, you'd beed to spend additional valuable times dping the same with Postgres.
This is why, when it comes to Architecture, I increasingly find my myself over-enigneering a bit. Assuming there is a good chance you might need to upgrade your architecture in the not too distant future, that approach is actually kind of very efficient. I find that I am able to uncover a lot of potential gotchas, which feeds back into the what the simplified current architecture should be, and helps me understand the roadmap I'm facing very well. I also avoid wasting too much time going too deep in directions that make sense now, but need a lot of plumbing to get right, when I can see that I'd likely have to throw it all out in a few years. Going from A -> B -C -> D, where each step is the optimal good-enough-for-now architecture but which requires a lot of work to stabilize and iron out the kinks of, is much less efficient than exploring D well enough to know whether you should build A, B, or C now.
Basically, some over-engineering, if done right, is not wasted. It cuts right to the heart of what you are dealing with, efficiently, and allows you to make (maybe) simpler but informed choices now as to how best to allocate your development resources now.
Don’t do N+2. The goal isn’t to predict the future, nobody can do that. The goal is a durable understanding of the domain and the best fit implementation you can get with that current understanding and resources.
That said, SQLite passes that bar for me in most use cases.
It sounds like sqlite sessions handles the hardest part and you've sanded down some of the rough edges while implementing the glue bits for s3 (generously licensed MIT); thanks for the heads-up!
A distributed WAL (to survive a machine death) would also probably be something I'd want, and … something I'm not sure you're getting directly from SQLite.
Google "Log-structured merge trees" if you want an interesting read.
"Sockets are all you need for durable workflows" and then finally "Kernel primitives are all you need for durable workflows."
But seriously, part of being a professional is using the right tool for the job.
One reason why a "logs are all you need" solution may fail: untrusted-log-as-injection[1].
Check those SBOM, and don't forget to include their CICD pipelines[2].
[1] https://news.ycombinator.com/item?id=48315440
[2] https://github.com/jqwik-team/jqwik/issues/708#issuecomment-...
Different workflows should probably go in different buckets or "topics" for clarity. Since it's distributed, the system must guarantee that the log items are stored in the same ordering ("offsets") among the nodes.
Not a bad way to do things.
I am joining a new project and need to know to what extent Kafka is still a part of the future for new big data projects. It doesn't seem like there are alternatives at the high end but instead the question is when other technologies (that are easier to manage, require less compute, etc.) max out.
Yes
> I am joining a new project and need to know to what extent Kafka is still a part of the future for new big data projects.
It's not gonna win or die on merit.
If I were to sit here and propose that you mutate bank accounts in-place (which I'm more or less doing by analogy right now over on https://news.ycombinator.com/item?id=48339103), and you just need safe enough locking technology to do so, I'd be immediately and rightfully shouted down that no one does that and you write down the money transfers and derive the final balances from that.
Manage any other kind of state, however, by appending state-changes and rolling them into a derived state, and then you're chasing fads and doing resume-driven development. So I'm skeptical about the future of this way of doing things.
I am finding that the most important thing is one big, consistent data warehouse that is updated with the state of the business as close to real time as we can get.
SQLite is not really great at this particular problem. Something like Postgres or SQL Server would be much more suitable for an OLAP data warehouse that can serve clients (AI agents) while simultaneously merging massive record sets from upstream business systems. These products also offer intricate permissions control. You can prove to an auditor that your AI solution will never see tables or rows it's not supposed to. SQLite doesn't even have a concept of a user, role or login.
> The compute can stay cheap and disposable.
Again, hosted sql is better aligned. The alternative is DIY hosted sql (SQLite + some other magic) which immediately violates this rule.
Funny how people are independently converging on similar patterns of "what works" here. Still feels like we're in the wild west with all these ad-hoc patterns of agent orchestration that people are coming up with.
Writes are single threaded, but, you can still easily do thousands per second.
DuckDB offers similar qualities, on the OLAP side.
This is not to say this is best combination.. but, when you consider the simplicity of setup, usage, operations, and backups, and cost element, this indeed offers one of the best, if not the best combination.
https://sqlite.org/datatype3.html
https://www.postgresql.org/docs/current/datatype.html
Working with date/time feels like using a 30years old database, nothing is enforced at insert. Really someone needs to explain why so many people like it.
create table events (
id integer primary key,
name text not null,
event_date text not null check (
-- YYYY-MM-DD
event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'
and date(event_date) is not null
and date(event_date) = event_date
)
);
sqlite3.IntegrityError: CHECK constraint failed:
event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'Python would show the 1st line always? Or the failed part?
This is unreasonable for a very common type I think.
The somewhat good: it gets rid of most of the weak typing. It still coerces, in line with other SQL databases, but at least a column will only store values of one type. Personally I’d prefer to opt out of the coercion. And I don’t think most ways of writing SQL (in applications especially, but also manually) will ever actually trigger the strict differences. So it doesn’t feel like it’s actually particularly useful.
The distinctly bad: you’re limited to six datatype names. You may well now want external documentation or load-bearing comments in your schema, and your application code may be hobbled, if it liked to infer types based on the datatype name. For example, in sqlx, SQLite datatype BOOLEAN can automatically map to Rust type bool <https://github.com/transact-rs/sqlx/blob/75bc0487eb661da811b...>. Without that, you have to resort to a variety of less-pleasant techniques, such as selecting `done as "done: bool"` or overriding things in sqlx.toml.
I really, really wish they’d implement some form of CREATE TYPE and let that work with strict tables. If I could `CREATE TYPE BOOLEAN FROM INTEGER` and such, I’d be all in on strict tables.
I agree it is disappointing, especially before strict tables.
You should check out DuckDB which is basically SQLite but with proper types. Although it is also OLAP (struct of arrays) rather than OLTP (array of structs) which may have worse performance for typical SQLite loads. In practice I doubt it matters if you have an application where you're considering either.
PRAGMA journal_mode = WAL
PRAGMA foreign_keys = ON
# Something non-null
PRAGMA busy_timeout = 1000
# This is fine for most applications, but see the manual
PRAGMA synchronous = NORMAL
# If you use it as a file format
PRAGMA trusted_schema = OFF
Also use strict tables. https://www.sqlite.org/stricttables.html
While it has bad ergonomics, you can also use CHECK constraints. For example, using sqlite's built in date support, it's possible but awkward:
CHECK (
date(my_date_col) IS NOT NULL
AND my_date_col = date(my_date_col)
)
New types would break forward compatibility in SQLite's terms. 3.7.0 added WAL mode was their example of a forward compatibility break.[1] 3.y.0 could add better type system mode.
Another fascinating fact: our countries TLD has been stolen Ocean's 11 style (I am not kidding). After Czechoslovakia split into Czech Republic and Slovak Republic, the newly created Slovak .sk TLD has been under the care of people from the local university. The university also had some offices that they were leasing out. Someone had leased this office space (EDIT: this is important as this means they had the same physical address), created a company that had the same name as the NGO that was taking care of the domain, so e.g. the NGO was named "My Company o.z." and the perpetrator created a "My Company s.r.o." (our countries version of the american Ltd). This person then wrote to ICANN to change the address to the "My Company s.r.o." presumably under the pretense that this was just an administrative error and from this point, they have functionally taken custody of the TLD. I was not able to find how they did it technically, but I presume they persuaded ICANN to then point to their servers instead of the real ones. After this happened, it seems that no one noticed for some time. When they noticed, they tried taking it back, but they weren't able to. For some inexplicable reason, the government during that time (Šuster era, early 2000s) gave the new company a contract that was functionally uncancellable from the government side. Later governments made this even more uncancellable and in 2017, then Minister of IT (and as of this day president!) Pellegrini made the contract literally uncancellable. As a result of this, we have one of the most expensive domains around (18e/year, rising each year for no good reason). (EDIT:) The company running our countries TLD is now a foreign entity that the whole thing has been sold to (multiple owners over time) and we as a country have no control over if I understand it correctly.
I might have gotten some details wrong as I am writing this from my memory of researching it a couple of years back, but you get the idea, crazy stuff. Here is an article in Czech [0] that tells the story a bit better, but you have to translate it.
[0] https://www.root.cz/clanky/pribeh-domeny-sk-aneb-kradez-za-b...
// EDIT: I have found that the article actually links the movement to return the TLD back [1]. It also has a story tab [2], so they have something much more precise than the paraphrasing I wrote.
maintain an in-memory SQLite db and work it with SQL commands, and if you also want to preserve state across application restarts you can routinely save to disk or load from it: <https://www.sqlite.org/backup.html#example_1_loading_and_sav...>
this also happens to be the most convenient file-format (aka. application-format) I ever worked with.
Obligatory list of workflow engines and libraries because it's such a common need that a lot have rolled their own. [1]
[0] https://docs.dbos.dev/python/tutorials/database-connection
The crux of all these “you only need sqlite” posts isn’t that you need SQLite, it’s that your architecture only ever needed flat appendable log file to begin with.
Once you realise that SQLite is just an aggregated view over this log it calls into question if you needed a third party query engine at all, suddenly the whole NoSQL vs SQL debate becomes a meaningless implementation detail.
Server based databases aren’t special beyond the fact they’re just distributed synchronisation primitives over files on a file system. But the utility of this is quite limited in practice, writing to a table will always require some sort of synchronisation, using SQLite vs Postgres is just a case of where you put the lock - in your own code or in a 3rd party server.
You are correct on the premise, however my observation has been that teams who try to be lean and mean and minimize dependencies always start off believing they don't need a query engine... and they always end up needing one, often making things much worse by refusing to just migrate to any database due to sunk cost fallacy.
I fully agree with your argument that it's all just file operations in the end. To me SQLite wins for everything beyond persistence however: the SQL commands, extensions, backups, changesets / patchsets.
Can you, I, and many others write something that has 10% of SQLite feature set that serves us perfectly? Absolutely! But I don't want to, and in business settings nobody will allow you to hand-roll crucial infrastructure software (like databases) unless they're in full crisis mode and you're the mega-expensive consultant brought in to save the day.
ALTER TABLE users MODIFY COLUMN…
ALTER TABLE users ALTER COLUMN…
ALTER TABLE users ADD CONSTRAINT…
2026-04-09 (3.53.0) - "Enhance ALTER TABLE to permit adding and removing NOT NULL and CHECK constraints"
I use my own sqlite-utils CLI/Python library to work around these limitations: https://sqlite-utils.datasette.io/en/stable/python-api.html#...
Check it out here: https://unmeshed.io
(I'm already using Postgres, so I don't really need a sqlite-based durable workflow engine, so looking to know how to choose between DBOS and Obelisk)
Think Wirecutter, not install guide.
Small, well defined, data model with known query patterns? Bespoke model
There probably is a place for sqlite and my project space so far hasn't yet well-aligned with it.
https://github.com/guilt/squeue
It did the job, was fairly easy to use.
Building durable workflows on Postgres
Armin Ronacher's "Absurd Workflows: Durable Execution With Just Postgres" https://lucumr.pocoo.org/2025/11/3/absurd-workflows/
[0]: https://pglite.dev/
Quote 2: "SQLite State backed up to S3". Yeah buddy, if you think S3 isn't Postgres I'll eat my foot.
Wow. Really? I thought I needed to use an overpriced cloud SaaS for everything.
In short: SQLite is not all you need, unless you’re just experimenting don’t actually care about durability, in which case you also need litestream + object storage.
Right.
I want to love it, and I don't take open source projects like this for granted. But during my last production upgrade I chose to decommission Litestream in favor of a dumber, less granular solution using sqlite3_rsync and nightly backups because there is no point in using a backup system that is not rock solid.
By default. Generally your primary database is in a completely different failure category than a kubernetes node running an ephemeral workflow pod.
synchronous_commit = on PRAGMA synchronous = fullThat's distributed workflows :)