Hacker News Reader

Comment by cynicalsecurity | Hacker News Reader

They came up with a really over-engineered, over-complicated attempt at splitting data into multiple databases. I'm not sure this was the best idea or even a good idea, but then I also don't understand how it came to the situation when they only had a few months before their current database gets overflown and the whole system collapses.

View on HN

Can you offer insight into what a better approach might have been?

As others have mentioned, moving to per tenant databases can really simplify things at scale and doesn't leave a massive amount of engineering complexity and debt in its wake.

I feel sorry for the team managing this 5 years from now.

Moving to a per-tenant database sounds like even more work to me than moving to shards. Moving to per-tenant means rewriting _everything_ - moving to shards has you rewriting a lot less.

What do you mean by "rewriting everything"? Or maybe your definition of "per-tenant database" is different from mine. In our product, it's just a small layer which routes requests to the target organization's DB. When an organization is created, we create a new DB. Most of application code has no idea there are different DBs under the hood.

There are logical DBs (folders/files on a single physical server), and there's a few physical servers. We're currently at the stage where the first physical server hosts most of smaller organizations, and all other physical servers are usually dedicated servers for larger clients with high loads.

If you didn't write your application with multi-tenant in mind from the start I would expect you would need to review almost every line of code that touches a database to make a transition like this.

In our code, the only DB-related piece of code which is aware of multi-tenant databases is the getDBConnection(accountId) function. Once you have the connection, you execute exact same SQL queries as before. The function is hidden deep inside the framework/infrastructure layer, so application code is completely unaware of it.

This is possible to do, but lots of engineering. You can provide the experience of a single DB while each tenant can be placed in their own dedicated Postgres compute. This would help the application to stay the same while tenants are moved to independent computes (you can even move only a few tenants and leave the rest on a shared Postgres compute).

Exactly this. But at this point, I don't even want to give them advice, I don't really like their service. I like Lunacy more.

Reply on news.ycombinator.com