Hacker News Reader

Comment by totally | Hacker News Reader

What would be more sane? Allowing things to remain broken at night? Having someone who is not a software engineer fix it?

View on HN

Either pay another full time night shift or yes, accept that things are broken for the time.

If your product is really so important that it can't be down, hire more engineers and pass the markup to your customers.

I'm glad I live in the country where you have to have 11 hours between end of work and start of work(except for special cases afaik).

There are legitimate reasons to pull in someone after hours, but it really has to be catastrophic. I'd 100% want to be called in if I deployed something knocking out 911 service for a whole state and I was the only one with the knowledge to actually fix it in a timely manner. However, most problems are not like that and are either able to be delayed until an actual business day or can be solved by someone else.

Let's be real, we're talking about line of business apps and ecommerce stores making $5,000/day total revenue. "Critical infrastructure" has an entirely different failure model.

If the product that breaks in the night is SO important for the company, well, why is not the company paying for dedicated people (not the engineers who create the product) to take care of it when it's broken? As said above, while on-call you don't write code, you just turn off feature flags, reboot machines, etc.

If the company cannot afford that, then the product is not that important and can remain broken until the morning.

Even 24h fast food places hire 3 people (each working 8h)!

If a 7-Eleven is open 24 hours a day, they usually hire three 8-hour shifts (roughly speaking).

If you want things to not break, have redundancy in hardware and failover modes that let you function in reduced capacity.

Manual fixes should never be done in a hurry, and if your system is that fragile, I really wonder about the competency of your senior employees and leadership.

Reply on news.ycombinator.com