Hacker News Reader

Comment by deltaci | Hacker News Reader

this is already the third time github actions is down this week at wednesday morning

maybe they have a "no deploy on Friday" rule :-D

I sure hope not, GitHub is supposed to be matured infrastructure at this point, where most if not all changes going into production should be very well tested and nothing that multiple people haven't verified as being correct should end up being deployed and released.

Besides, Microsoft surely has 24/7 watch of their infrastructure, even on weekends, it's a huge company.

Bugs are a function of change, not a function of maturity.

Just because they can have people come in on weekends to fix things doesn’t mean they like doing that.

I know many “mature” software platforms that do not deploy on Fridays or off hours at all

A true global company doesn't have off hours.

As someone who runs one of the most global APIs in the world, I promise you, I do in fact sleep

That's why a global team is important, when you sleep they work, when they sleep you work. Work is constant when you're servicing the globe.

And work is constantly slow as well, since it’s impossible to get people in the room at the same time

Why does everyone need to be in the room? I have a groomed backlog and can talk to people async as needed. We also record meetings if you missed them and depending on the context of the meeting or importance, we'll hold timezone friendly meetings for everyone as required.

¯\_(ツ)_/¯

We have a different work philosophy. I do work with teams in India and England, and it’s painful to accomplish anything cross team

Everyone employed by your company, who work in the same industry stops and starts working at the same time, all around the world?

No, but my team owns our own APIs. We are all located in the same area and we go oncall for our service.

What's your pager rotation like? I want to say you have follows-the-sun, and so your on-call shifts are 12-hours long and you swap with a team on the other side of the world from you so you can get said sleep, but I don't want to just assume that.

Dayshifts are 9-5, night shifts are 5-9. Same team rotates through both. I have done plenty of overnight oncalls.

Some teams do follow the sun type rotations, but my team is all in Seattle.

Fascinating, glad I asked!

Less "on" hours then? Even Google has diurnal patterns when there's a lower amount of traffic simply due to the fact that humans are unevenly distributed across the Earth's surface. And Google does code freezes for the holidays where they don't deploy at all.

> Besides, Microsoft surely has 24/7 watch of their infrastructure, even on weekends

"watching" with a dedicated team vs "waking up everyone in engg because things are on fire" are two very different things.

Besides, size doesn't work that way. The larger the organization and the more complex the product is, the higher the chance some unexpected interaction will occur. There are processes and automation that can mitigate this, but one can never be completely certain.

Not even the aviation industry has mastered that.

Reply on news.ycombinator.com