Comment by kentonv | Hacker News Reader

Comment by kentonv | Hacker News Reader

Parent Full thread

Personally, I'm a firm believer that most "web app" use cases are better served by many small databases (e.g. per-user or per-document) rather than a single monolithic databases. This is especially true when serving users all around the world -- per-user databases can be located near each user (both for speed and to comply with data locality laws).

What I'd like to enable here is a progression where you start out prototyping your app with a single D1 database, which is easy to use and reasonably fast. Then as you grow we provide tools to let you transition to many D1 databases sharded in a way that makes sense (e.g. per-user). Apps that want even more control can move to using full-on Durable Objects (which will soon support a SQLite database per-object).

That said, there are certainly many use cases out there where simple monoliths make sense, especially non-interactive data crunching. I'm not sure yet if D1 will ever be the right choice for those, but the Workers platform aims to provide many options.

Thanks for the insight, I greatly appreciate it! This definitely is a reasonable idea for many things and I'm looking forward to seeing something similar to the sharding mechanism in the future.

I've only started to think about this and I'm thinking the hardest part will be dealing with cross-cutting concerns (in a non-auto sharded world manually creating multiple database) and trying to find a way to keep each database isolated without extra burden compared to using a hosted Postgres.

As an aside, that lan optimized house was a gaming dream. Hope your new house is as awesome.

I've been working on a little project that uses D1 and have been hoping that is how D1 will evolve. A db per customer in my case. Is the colocation in a durable object so 'business logic' and sqlite can live in the same isolate for performance and security? Would that be a post 'out of beta' feature? Ive been building as a monolith thinking that the number of databases in the alpha was indicative of the future.

" Apps that want even more control can move to using full-on Durable Objects (which will soon support a SQLite database per-object)."

Can you elaborate this little bit more? Im using DO today and i have a bad time sharding my data (works, but i hate it);

So i will have the option to use the standard store or/and SQLite?

If so, i dont can keep with my DO (because i have control of everything) and use SQLite for things that is bigger than what the value store supports.

Sorry, I don't quite understand what you're asking.

In the future each DO will have a private SQLite database. The key/value store will actually be redirected to store into a special table in this database, but probably new apps will just use the database and not the KV store.

Separately from that, I would like to develop tools that make sharding Durable Objects (and D1 databases) easier. Today it's a pain to do manually. This is independent from the underlying storage model, though.

Reply on news.ycombinator.com