Hacker News Reader

Comment by dilyevsky | Hacker News Reader

Afair Google just ran irc on their corp network which was completely separate from prod so I wouldn’t be surprised if it was in a small server room in the office somewhere.

> very broadly applicable. I don't see any "this would only apply at Google" in here.

One thing I haven’t even heard of anyone else doing was production panic rooms - a secure room with backup vpn to prod

View on HN

When I was there, the main IRC ran in prod. But it was intentionally a low-dependency system, an actual IRC server instead of something ridiculous like gIRC or IRC-over-stubby.

i think it had a corp dns label but i'm fuzzy on that. yes it could've been a prod instance which would mean you'd need to go to panic room in case corp was down but maybe that was the intention.

> Afair Google just ran irc on their corp network which was completely separate from prod

I thought Google didn't have a "corp" network because of their embrace of zero-trust in BeyondCorp?

I don't think zero-trust prohibits network segmentation for redundancy or due to geographical constraints etc. It's mainly about how you gain access.

Correct. At an old job we did zero trust corp on a different AWS region and account. The admin site was a different zero trust zone in prod region/account and was supposed to eventually become another AWS account in another region (for cost purposes).

I can’t say if any of this was ideal but it did work unobtrusively.

They do. But I'd say most employees go their whole career without needing to do anything that requires a VPN.

It's basically all web based access through what is, at the end of the day, a http proxy.

SREs need to be ready for stuff like "hey, what if the big proxy we all use to access internal resources is down?".

Way back when, for a while, our local (Google) office's internet access ran off the same physical lines as the local prod datacenter traffic. So, any time there was a datacenter traffic outage of any kind, our office was also out. There weren't a lot of outages of that variant, but we knew immediately when one was happening. It's not particularly fun to have all of your access go out concurrently with a prod outage.

Reply on news.ycombinator.com