Hacker News Reader

Different mindsets. I was cutting my teeth on a set of 12 static servers/processes, with no autoscaling or auto reboots. Memory leaks were rare but reported. Usually FD leaks were culprits. So learned a ton about how to write really correct applications, and today it just feels like none of that matters. And in complete fairness, with today's ops tools, it probably doesn't.

View on HN

At scale, server failures are going to happen. Maybe it's a power outage. Maybe it's a failed drive. Maybe it's a kernel crash. Only one of those can be remedied (not even prevented) with careful debugger investigation, so you're going to have to do the work to be able to handle such failures gracefully. At that point, unless the leaks are causing outages so rapidly that the machines can't handle, then rebooting becomes not that big of a deal, while wasting senior eng days trying to track them down is a real cost.

There are some nasty escalation scenarios I’ve seen in that space. Cost increases as does workload backlogs and latency.

Eventually something breaks and causes an outage.

All leaks should be resolved lest you end up with floods.

Reply on news.ycombinator.com