Hacker News Reader

Comment by peter_l_downs | Hacker News Reader

What do you mean? I don’t understand, how is using a database an architectural mistake?

It's a mistake to use one specific computer science concept (RDBMS) to solve 50 different problems. They mentioned logging and scheduling, two things RDBMS are not designed for and have specific limitations around. From just a general architecture perspective it's literally a single point of failure and limitation for every single aspect of the system. And it's vendor specific, it's not like you can just plug plsql code into any other RDBMS and expect it to work. It's so obviously a bad idea it's hard to comprehend taking it seriously

You'd probably be surprised by how many systems are just Postgres/mysql + Redis.

For example, it's dead easy to make a high-capacity message queue by just using SELECT ... FOR UPDATE SKIP LOCKED with Postgres transactions, and I would argue it's more reliable than a lot of microservice-everything setups by way of having very few moving parts.

Classic NIH syndrome. "I made it myself so it must be better", when it's clear that a single sql query doesn't remotely approach a complete solution for scheduling. But the ignorant use it because they don't know better, until they too fall into the trap and realize they spent 10x as much engineering work to get something they could have just installed from the web and been done with. Every generation seems to fall into this trap with another tech stack.

If you already have a DB (and essentially every app does), it can be far less effort with the same or greater reliability to create a queue table than to set up RabbitMQ, NATS, etc. As long as you tune the vacuuming appropriately, it will last for quite a lot of scale.

Source: am a DBRE, and have ran self-hosted RabbitMQ and NATS clusters.

It's all trade offs, right? Introducing a new component to your stack isn't free, it's paid for by more operational complexity. Maybe it's worth it, maybe it's not, but there is a calculation that needs to be made that's not just "NIH syndrome".

Sure, so install RabbitMQ as well.

As the saying goes... "now you have two problems". :)

You could honestly just do in memory SQLite and use that lol idk that’s what I did because I wanted to quickly be able to handle thousands of simultaneous scheduling tasks.

Took like two hours and it works fine. Customers are happy. Event logs persist to s3 in case I need to replay (hasn’t happened once yet).

It might not be good computer science to use one tool to solve 50 different problems; but it's not bad computer engineering to use one tool to solve 50 different problems that fit within its capabilities rather than using 50 different tools, all with their own operational expertise.

There's no need to have the best tool for every job. Although it's also important to be able to see when a many purpose tool is insufficient for a specific job as it exists in your system and then figure out what would be more appropriate.

Reply on news.ycombinator.com