Starbase: SQLite on the Edge
starbasedb.com
starbasedb.com
I wonder what are the use cases (and when it's safe) to use "eventually consistent".
I'm guessing that maybe things like social media posts could be fine with "eventually consistent". It's not really critical to have the latest data.
I'm guessing that things like a shopping cart, a user account, a session shouldn't use a "eventually consistent" database.
Is it auto-accessible REST endpoints? Easy to implement websocket support for the database? Data replication for scaling reads? Offline data syncing? A lot of potential wins for a layer like this to build on.
Or, we have easy to use, enjoyable interfaces and let ourselves use them, and actually finish things.
DO implements a very comprehensive strategy to mitigate data loss in DO backed by SQLite. But Starbase seems to add a layer of query caching on the application level, which seems to be naively implemented (I might be misunderstanding, please correct if I'm wrong).
[1] https://starbasedb.com/blog/developing-acid-transaction-supp...
Learn more about that here: https://github.com/Brayden/starbasedb/issues/12
Update: bug in redirection: https://starbasedb.com/blog/blog/blog/blog/blog/blog/develop...
D1 = Cloudflare D1, https://developers.cloudflare.com/d1/
Just incase someone else was as lost as me.
Zero-latency SQLite storage in every Durable Object
D1 is SQLite
https://blog.cloudflare.com/sqlite-in-durable-objects/
There's actually a section called: What about D1?
Pretty excited about this tech, it really flips the script on the concept of a DB per user / sharding approach which used to be costly at the schema/physical db level.
What is the architecture overview? Does each worker have a copy or do all of them work on the same DB files? How does it handle concurrent workers and db locking? Is SQLite native locking mechanism sufficient in this application?
We probably need to build out some sort of data replication to scale reads though Durable Objects are limited to 1000 RPS.
I have this old school expectation that I should be able to understand what your website/product/project is about but reading the first few sentences in the "above the fold" section of your homepage. I don't want to play a game or click below the fold "read more" links to try and find out what this web content has to do with the domain name "starbasedb".
Perhaps they actually know their intended (seemingly game developer?) audience that I give them credit for? But I do have plans (admittedly only very vague plans right now) to use one of the available "SQLite from javascript in a webpage querying the SQLite file in an object store like S3 without a web backend" techniques or projects. Maybe they'd work for me with my non game oriented use case? I don't know, it seems like they don't want me to find out...
As far as scaling yeah there are still some challenges there. I mentioned in another response there's a limit of 1K RPS, and the single-threadedness could become a bottle-neck. I think there are probably clever workarounds here that we still want to explore, for instance each Durable Object could handle a seperate piece of state and you would effectively have a level of state-based sharding.
Last time I saw that was zendesk with their Buddha's and weird tonal audio devices.
Edited for corrected semi-colon because I'm haunted by a grammar-nazi.
I'm not sure if I misunderstood something, but using a global queue/lock to enforce isolation properties seems to be just a little disingenuous for a dbms. It works, but it's odd to talk about isolation in a system with zero concurrency. It's like saying a bicycle gets infinite miles per gallon in fuel efficiency.