Show HN: WunderBase – Serverless OSS database on top of SQLite, Firecracker
wundergraph.com
wundergraph.com
It's a nice blog post about gluing technology and I can see how this could be a really nice way to run some lower-cost databases in a non-demanding development environment. However, it is not a reliable way of operating a database. For me it isn't really serverless since it only scales between 0 and 1 instance whereas a serverless DB ideally would scale-out but should at least have some ability to scale to greater load in response to demand, along with higher reliability and availability, and backing up data to object storage.
It would be cool if you could ditch the graphql layer. It seems like there are other alternatives that go the vfs route so you still get to use a standard sqlite client.
But I mostly agree that we need a better term than "serverless" for this kind of stuff. The big things people seem to want from "serverless" solutions are "not managing long-running server instances" and "true usage-metered billing".
Whether or not there are servers, like, at all has not all that much to do with things.
When does SQLite become "a distributed DB built with sqlite as storage"? Did Postgres stop being Postgres when someone plugged log shipping into it? That's basically what we're talking about here --- not stuff like rqlite, which I'm also pretty interested in, but which really is a new database built on SQLite.
Cockroach is not the same thing we're talking about here; it's a much more ambitious design, just like rqlite is much more ambitious than shipping SQLite transactions. What we're talking here is the tooling needed to generate a single-writer multi-reader cluster the way you would for Postgres, but for SQLite instead. I don't know if single-writer multi-reader clusters for Postgres qualify as "easy", but they're not science projects.
If it's not obvious: we love Cockroach. Our commercial bias is that we built a platform that is especially useful for distributed services and clusters, and Cockroach is very much that.
Still, I would expect the windows of read staleness and data loss to be much wider with an approach of just shipping logs rather than re-engineering things like RQLite, etc. Trying to ship a copy of SQLite to every app will mostly achieve lower latency at the cost of staler reads, although I can see how ideally it could cut the latency in half by streaming data ahead of time. But of course it increases storage costs dramatically as things scale out.
And there's a distinct advantage to doing this with SQLite: if you can viably deploy SQLite (because your tooling is asymptotically approaching the state of the art for Postgres), your reads are ultra fast, because you're not hitting the network for an extra round trip every time you query your database.
I think that AWS already offers Aurora Serverless v2 which pretty much accomplishes what a lot of these me-too-serverless services that won't integrate as well as something that is offered out of AWS.
Even if you were insistent on cloud-agnostic mandate (which is really not logical since there is at best 3 public clouds to choose from that are also vulnerable to targeted cyberattacks and faultlines), it would be hard to convince a large organization to switch to using Sqlite on Fly.io
Reason (3) is clearly our ulterior motive here, so we're not disinterested: our model user deploys a full-stack app (Rails, Elixir, Express, whatever) in a bunch of regions around the world, hoping for sub-100ms responses for users in most places around the world. Even within a single data center, repeated queries to SQL servers can blow that budget. Running an in-process SQL server neatly addresses it. Conveniently, most applications are read-heavy, and most performance-sensitive app requests are reads.
tryna wrap my head around this architecture, it is quite interesting but concerning that it is now sharding into close-to-local sqlite instances located near the user.
(You're not generally "reaching back to the central source of truth to compare" things, so much as "satisfying the write centrally and shipping out the new database pages back to the read replicas at the edges").
More on this model: https://fly.io/blog/globally-distributed-postgres/
Are there cold start delays? From the moment I type domain.com is it going to spin up a fly instance closest to me and serve the SQLite database reads?
I'm gonna give this a go this weekend to see what it can do
People should just be aware that Aurora Serverless v2 won't scale to zero, and you'll pay for it even if you never use it.
https://www.lastweekinaws.com/blog/no-aws-aurora-serverless-...
> Prisma is an API server that puts a GraphQL API in front of a DB.
Prisma is an ORM which generates a JavaScript/TypeScript client library for your database.
Your description is very true for Prisma 1 (which has been in maintenance mode for several years and is officially deprecated by now [1]), but the latest version(s) of Prisma (v2+) don't expose a GraphQL API any more. Prisma 1 also used GraphQL SDL for data modeling, the Prisma ORM on the other hand has its own, custom modeling language for describing database schemas in a declarative way and also comes with a flexible migration system.
That being said (and as Jens also mentioned elsewhere), the Prisma ORM does use GraphQL _internally_ as a wire protocol. However, as a developer, you _never_ touch this internal GraphQL layer and are not even supposed to be aware of it (you actually have to jump through a lot of hoops to even "find" it). It's also very likely that we'll replace GraphQL as a wire protocol in the future, so "GraphQL" really isn't something you should be thinking about as a developer who is using Prisma.
Hope that clarifies the situation a bit, let me know if you have any further questions around this topic.
And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language?
To be honest, I think the most comprehensive resource for this is a Twitter thread [1] that I posted from my personal account that explains the different turns we took in the product direction :D
Otherwise, there's yhr blog post "How Prisma & GraphQL fit together" [2] that also touches on the same topics but might be a bit dated since it was published before we released the Prisma ORM for production use.
> And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language?
The short answer is that it would allow us to make us more optimizations for the bridge between JS- and Rust-land in Prisma Client. However, this is not an urgent issue for us at the moment so very likely won't become a priority in the near/mid-term future.
[1] https://twitter.com/nikolasburk/status/1384908813069869058
[2] https://www.prisma.io/blog/prisma-and-graphql-mfl5y2r7t49c
- https://github.com/uktrade/sqlite-s3vfs (Read/Write) - https://github.com/michalc/sqlite-s3-query ( Read Only)
I've been looking for something similar for some time to use in my development docker instances (specifically with dokku). I have many services that, although they consume little CPU time, they do have a high overall consumption of RAM, but they are actually used for a few minutes each day.
I don't want to use kubernetes for this as it adds too much complexity for the benefit I would get.
Do you know any solution similar to this, to turn on / off docker containers when network traffic comes in?
People have been sleeping on SQLite and are starting to wake up and I'm kind of psyched to see what else they come up with (another very cool example of a software tool that really plays to SQLite's strengths is Datasette: https://datasette.io/).
https://github.com/pocketbase/pocketbase
PocketBase is an open source Go backend, consisting of:
* embedded database (SQLite) with realtime subscriptions
* built-in files and users management
*convenient Admin dashboard UI
*and simple REST-ish APIAFAICT because you can only update a row via the UI, it's easy to tell when a row is updated.
But if you're talking about raw SQLite, there's [1].
The Realtime API is implemented via Server-Sent Events (SSE). Generally, it consists of 2 operations:
* establish SSE connection
* set client subscriptions
Pocketbase is mainly a single application with SQLite embedded, so, whenever you update records through PocketBase's API, the main application always knows that.
This is different with postgres, since it may be connected by several different applications/servers remotely.
My SEC team felt a disturbance in the force from me even considering this on our internal network. Security should not be a secondary consideration for a DB!