Hacker News Reader

Great question!

Depends on your workload size, if the compute time is less than time it takes to transfer the data back and forth, then it might be a bad idea to use it indeed.

Typically these solutions should only be tested after maxing out vertical scaling, before applying for horizontal scaling.

Network is one hell of a destroyer when it comes to advantages gained from distributed computing.

View on HN

I would go even further and argue that for the vast majority of organizations vertical scaling is all you are ever going to need.

horizontal scaling however provides you with more aggregate network bandwidth. Most enterprises run workloads that downloads data from a data lake (usually S3), which is usually the bottleneck. Having horizontal scaling here allows the query leverage much higher network than just having a large single machine.

Reply on news.ycombinator.com