Comment by msisk6 | Hacker News Reader

Comment by msisk6 | Hacker News Reader

This is one of the better articles I've read about the grid.

I don't work in this region, but I work for another ISO that controls one of the other grids doing a cross between DevOps and SRE on the systems that directly control the grid.

I've worked for startups (including my own) and public cloud providers, but this is the most stressful job I've had. If I fail people will quickly start to die as traffic lights and life-support systems fail.

The article touches on the complexity a bit, but it's huge job controlling a deregulated open-market energy grid. You have to scale for peak demand while providing financial incentives to support a mix of generation with widely varying prices. You have to deal with distribution congestion and natural disasters. Even generation disruptions -- a 2-GW nuke plant can shutdown at any moment. You need to deal with that deficit and quickly get something else online to meet demand.

And it all has to work 24/7 without a second of interruption.

It's an eye-opening experience. I'll never complain about paying my electric bill again. ;)

> as traffic lights and life-support systems fail.

Has anyone on life support ever died from the power going out ? I've worked at a hospital and the life support machines have internal battery backups, they're plugged into a separate outlet that is (at least) got an instant-on backup generator and possibly another battery backup. Plus, the ventilators I saw have rubber balloons on them that can be manually squeezed to breathe the patient if the machine dies. I also thought ventilated patients are also staffed by critical care nurses that have at most 2 patients per nurse. I've never heard of a ventilated patient set off in a room alone without adequate supervision.

Not saying there's no risk elsewhere in a city from power loss, but I thought hospitals had this situation covered.

Probably multiple in New Orleans during Katrina. Of course, that was more than just a power outage, with flooding water destroying backup generators, more critical needs patients than non-evacuated staff remaining to support them, multiple days for support to arrive, etc.

This podcast contains a horrifying story about how life-supporting services and resources were managed when one hospital in New Orleans lost power during Katrina: http://www.radiolab.org/story/playing-god/

Things are not always properly maintained, human error happens, back-up equipment fails, etc. In hospitals I am sure it's very rare but I'll bet it happens. But not all people on life support are in hospitals...

http://www.timesunion.com/local/article/Nursing-home-death-d...

People die when the power goes out. :) It can be both accidentally and when people's power is cut off due to non-payment.

They also die from heat exhaustion when the AC goes out.

http://www.pressreader.com/new-zealand/the-new-zealand-heral...

If we fail at the grid management level and the entire grid collapses, it's a non-trivial process to get it back up. It can take hours to days to get everything up and running. It's a process called blackstart.

In the meantime hospitals and other facilities with backup gensets will quickly run out of fuel. Those fuel delivery contracts everyone has will go out the window once the entire grid is down and everyone needs fuel.

Your backup-electricity strategy has to be decently thought out. The Fukushima disaster would have been avoided had all of the switching stations for the backup generators been located in safe, or even elevated, structures[0].

[0] https://en.wikipedia.org/wiki/Fukushima_Daiichi_nuclear_disa...

Which ISO or vendor do you work for?

ERCOT in Austin, TX.

Awesome! I've worked with y'all before. Great engineers!

Reply on news.ycombinator.com