Turning off electrical grids to prevent wildfire− a complex, technical operation | Hacker News Reader

Turning off electrical grids to prevent wildfire− a complex, technical operation

theconversation.com

45 pointsrntn41 comments

I would like to know why grid operators turn portions off instead of installing devices to mitigate the fire risk when a fault occurs. “Rapid Earth Fault CLearance” devices (REFCL), also known as ground fault neutralizers, are a newish but commercially available technology that massively reduces current, and thus heating energy, delivered into a ground fault. In testing in Australia, a REFCL-protected line was intentionally dropped into dry plants without setting them on fire.

What gives? I doubt these devices are so expensive that they’re worse than the cost of “public safety power shutdowns” or of the billions of dollars that utilities are spending, and charging customers for, for undergrounding of lines and extra tree work.

> I would like to know why grid operators turn portions off instead of installing devices to mitigate the fire risk when a fault occurs.

The stuff you're mentioning is - at least from a quick Google, I'll admit that - only certified in Australia. Even here in Germany, introducing that onto the grid would take years for certifications. Bureaucracy moves slowly.

Then, you'd need to upgrade a lot of transmission lines including the regional distribution grid (i.e. < 100 kV), a single unprotected line in a danger-prone area is enough to start a deadly fire. And there are a looooot of power lines that haven't had an inspection for decades - the hook that caused the 2018 fire was almost a century old [1]. And so, instead of inspecting, maintaining and upgrading the many thousand km's of national and regional lines, shutting them down during peak fire condition times is cheaper.

IMHO it only makes sense to install REFCL devices as part of a general overhaul of an entire transmission grid area.

[1] https://www.wsj.com/articles/this-old-metal-hook-could-deter...

Because they already have the ability to turn portions off. Installing a lot of new equipment means regulatory effort to get a plan approved, and a lot of installation labor.

I imagine these systems will be put in place over time, but I was just looking at Wikipedia on Reclosers and they said Sensitive Ground Fault protection reduces the risk by 80%; if the risk is high enough, that probably still warrants deenergizing risky lines at times.

These aren’t quite the same thing. Sensitive ground fault protection attempts to shut down the line more quickly in the event of a ground fault. A ground fault neutralizer attempts (and apparently largely succeeds!) at reducing the fault current to near zero during a fault even before anything is shut down.

These technologies can be combined.

Electrical grids are massive, like almost unbelievably massive. Ownership is complex in that segments of it are controlled by different parties/companies with some of those segments having shared ownership. There only three grids in the USA and (I think) five total in North American with every device operating on a given grid have the capability to play by the grid’s rules and idiosyncrasies. Grid operators are constantly testing new equipment but it gets tested within sandboxes for years prior to adoption. Even if testing goes well, you still need individual buy-in from equipment operators who are allergic to risk and hate change.

I am not trying to say this is the right or wrong way here, just kind of the reality of operating such a complex network.

We had a fire near Boulder CO that destroyed 1000 homes and had a few fatalities. If there are 100mph winds and dry conditions then it just takes one spark to create an unstoppable fire.

The local utility closed down some (but not all) lines for a recent wind event and everyone is kvetching.

Maybe this isn't obvious but domestic electric service is a marginal business. It's not profitable for every customer. That's why it's a regulated monopoly. In edge and rural areas the lines are not profitable (you can't charge more to joe farmer at the end of hundreds of poles.) New subdivisions with new poles/transformers etc. cost less

Windswept lines and poles out in the middle of nowhere are the places at risk and "make no money." The utility doesn't "pay" for the shutdowns but they can get sued for a fire.

The thing that was most unclear with this incident in Colorado was: How did they decide which areas to shut down? It didn't make me feel like the people in charge knew what they were doing, when we left our neighborhood where the power was intentionally turned off (for over a day) to go to our friends' house where it was kept on, despite them having much stronger winds both in the forecast and in actuality. Their house is also in an area with much higher fire risk, surrounded by dry windswept vegetation. It's hard to trust an organization that says "we're turning off your power due to your area's unique risk of fire" when that conclusion seems incomprehensible to anyone aware of the weather and geography of the area. It's very hard not to wonder whether they are ignorant or are not being forthright. Maybe there is some information that would show that they made the right decisions, but if so, they certainly haven't shared it yet!

Agreed this was confusing. It seems like the risk factors that would lead to turning off a neighborhood (non-buried power lines, exposure to wind and available fuels) should have been known and announced in advance.

I'm in Lafayette, and it seemed strange that our power was on when neighborhoods in Louisville and Erie were out.

Yep, and pretty much the entirety of Boulder preemptively turned off, but not the areas south along 93 and to the east into Arvada near Rocky Flats, which are the exact super dry and windy areas where the Marshall Fire began.

I actually do think there must be some plausible explanation, but it's hard not to feel sympathetic to the people who are convinced it is a tit for tat by Xcel, targeted at people who have been involved in recent legal confrontations with them. There are good reasons they aren't being given the benefit of the doubt here.

I honestly worry more that truth is they were just winging it without any strong process or data with which to make the decision, and that this is their plan moving forward.

But the good news is that the PUC investigating what happened is the way to get these answers, and is exactly what Governor Polis is pushing for.

My power wasn't turned off - I did get the "we might have to turn your power off" warning though.

I live in probably one of the worst areas for wind, and had gusts to 100 that Saturday night - didn't get any sleep!

I had studied the local grid a bit for some other reasons a while ago, here's how I would put the pieces together:

My neighborhood is all buried lines, and everything is buried starting right near our substation. No raised distribution lines to worry about. However, it's still not really distribution lines that are going to cause wildfires, it's the transmission lines that go through the dry forest. Wind taking out a distribution line isn't that big of a deal, and it wouldn't cause much downstream fire risk.

My co-worker who lives about a half mile from me was without power for nearly 2 days. In other power outages, we've compared notes. Even though they live closer to my substation than I do, they're on a different substation.

The transmission line that powers their substation goes right through the mountains. It's really close to lots of dry fuel and the towers are very far apart, so it would be easy to get those lines to swing in the wind. Additionally, the way the winds blow, if their lines did arc and cause a fire, the wind would sweep it right into town.

So you really don't want to look at "why was my neighborhood without power, we didn't get much wind." You want to look at "where does my substation get it's power, and would an arc on that transmission line be catastrophic".

At least in my immediate area, the substations that were shut down were exactly the ones I would have shut down just by looking at a map and saying "if it's windy and dry here, do I want an energized line?".

I don't know about the decisions in the wider Front Range area, but I would be willing to bet that 1) it's a pain to de- (and re-) energize lines, so they don't do it without weighing the risk, and 2) much of our transmission lines out here are in the mountains to the west, which is terrible when you have westerly winds that will blow the fire into town, so they have a high risk weighting.

Great comment!

I'll look this up myself too, but do you have things I could read on the relative risk level of transmission vs distribution lines? They seem risky in different ways to me.

At least in the case of what (possibly) started the Marshall Fire, I thought that seemed like a distribution line that was knocked down, not a transmission line that arced. But that could be wrong (and of course it's disputed that it was a cause of the fire at all).

> At least in my immediate area, the substations that were shut down were exactly the ones I would have shut down

First of all, how did you determine which ones were shut down? A big part of the problem, in my view, is that information of the form "this is what we're shutting down, and why we think it's high risk" did not seem to be forthcoming anywhere. To put it more broadly and with a finer point to it: there was no communication and nobody, including, apparently, the municipal emergency services organizations, had any idea what was going on, or why.

But secondly, from cobbling together information from their website, gossip, social media, and driving around that day, my impression was the opposite of yours, for my surrounding area. My impression was that the pattern was extremely mystifying.

But sure, maybe the communication issue is the first order problem to solve. I certainly hope you're right that they based the decisions on a good risk analysis. But if so, they didn't tell people what that was. And lacking information on that, few will be willing to give them the benefit of the doubt that you're giving them on this, even if they deserve it.

And I don't just mean communication in the period of time just before during and after the storm. Did I ignore an email from them in the past few months announcing that they are planning on incorporating emergency preemptive shutoffs during this fire season? I don't think so! (It's also weird to me that they seemingly didn't need to make the PUC aware of this ahead of time, with an announcement like that being required as part of approving that plan; it isn't only Xcel that I think messed up here.)

It's easy for monopoly service providers to come off as arrogant and aloof, and lose peoples' trust!

I hope this isn't getting too local and for HN, but...

"which substations were shut down" - I used to work for a company that did a prototype for image recognition to identify maintenance issues for the power company (vegetation encroachment, towers in need of repair, etc.) So I had access to the GIS for this area and memorized it. So I can tell by which houses are out which substations went down. Also related to that project, I do know that people (at least on the engineering side of the decision makers) do understand risk and alarm fatigue and all of that. So maybe based on my experience, I'm giving them more of a benefit of the doubt than I should be.

"communication" - communication wasn't great for THIS weather event, I got one email from them that "might" have to do something. But looking in my voice mail archives I see a call on 3/2 and 3/4 and also accompanying texts and emails that they might have to shut down for the same reasons (there were winds then too). Maybe they over communicated the first time? I didn't know that emergency services weren't told, that's weird. If I were pulling the plug, you can bet I'd be calling all of the affected Sheriffs.

"marshall fire" - Hope I don't get too conspiracy theory here. I live right by the ignition point, so I know that area well. I think the cult nearby was the real culprit, looking at the barn and everything the day of the fire. The Sheriff's investigation said that they did have a (legal) fire a few days before the event and did everything that they were supposed to to put it out and cover it, so there's no way they could do criminal charges. It's just the 100 MPH winds were able to start a few embers, and then the embers just hopped buildings over the ridge.

I know the distribution line by 93 was also considered an issue, but it never really looked like it sparked anything when I went by. I did talk to a deputy at school pickup and he was giving me some inside info about sneakiness on Xcel's part, but I don't think you could really pin this on them.

If you know the area, you have the barn from the viral video and houses all along the north side of Marshall road, and then you have the mesa along the south side. Granted, the mesa did burn, but it was just a "grass" fire, the houses along the north side were just catching the embers from the previous house and it was a chain reaction the whole way down. I ride Marshall mesa all the time, and I never saw any real evidence of ignition down by the power poles that they were blaming. Of course, you still have at least 20kV in those distribution lines and there could have been a spark from those 100MPH winds, but I still doubt it.

Back to the main topic...just going on what I know about the grid layout and the weather patterns here, they shown down the substations that I would have. And communication could have definitely been better. I talked to one guy who lives up in the mountains. After 24 hours without power, he called Xcel to ask when they were going to turn it on, and they didn't know his area was out (due to damage, not voluntary shutoff).

Also, when a live power line goes down, they get a signal that's pretty accurate as to where the break is. When a dead power line goes down and then gets damaged later, they don't know where it is and have to put eyes on every foot of line before they reenergize to make sure it's safe.

It's definitely too local for HN, but it's a good conversation so I don't care :)

> I didn't know that emergency services weren't told, that's weird.

Sorry if I implied they weren't informed at all, the reporting I read was that they weren't informed with enough certainty or lead time to sufficiently prepare.

And I think this also goes back to what I think is the deeper point that this shouldn't be an hours vs. days thing, but a months to years thing. If the monopoly utility for a region has concluded that wildfire risk has become so high that they need to start doing large-scale preemptive de-energization to safely manage it, for the first time ever, I think that should be highly publicized and closely coordinated with the municipalities and emergency services involved. Again, maybe this all happened and I just missed it, but I have looked and not found anything about it anywhere.

For instance, the stoplight situation was dire. Perhaps some of the stoplights on roads where the speed limit is 65 could be upgraded with a backup power capability, if there were some lead time on this? I dunno, maybe that's totally impractical for an outage that lasts over a day.

But that's another thing... Is it really acceptable for a discretionary outage to last for almost a day after the weather event has cleared up? I understand that it's very challenging to reenergize dead lines, but it's hard to be as sympathetic to that when turning those lines off in the first place was a choice, rather than a result of the weather itself.

> Also, when a live power line goes down, they get a signal that's pretty accurate as to where the break is. When a dead power line goes down and then gets damaged later, they don't know where it is and have to put eyes on every foot of line before they reenergize to make sure it's safe.

To me, this reads like a very good reason to not de-energize lines unless it absolutely cannot be avoided! And that's the thing, at the bottom of all of this, I feel like, I've lived here for decades, and it will be difficult to convince me that "preemptive power outages absolutely could not be avoided" was true for this storm, but not of any of the others like it that I have experienced in that time.

Another thing I just thought of...how many times does a power company need to communicate to first responders about shutting down power? It goes the other way a lot - first responders asking for power to be cut because of a fire or an accident or something. So I bet they have lots of policy and procedures around that, but going the other way, I wonder if they just don't have it all written down yet.

Yep, agreed. I'm sympathetic to a large amount of what I'm saying here just being growing pains as they figure out how to do a new thing. But IMO, monopoly utilities, and especially their regulators, have a responsibility to figure more of this stuff out beforehand.

People can't say "I'm not happy with how you handled this, so I'm switching to your competitor", and that necessitates a higher level of scrutiny and responsibility.

Your assumption is that the outage is about the houses that are dark and not some line of silent poles/towers stretching beyond where the people are. The N Boulder outage was probably for rural lines N of Boulder rather than any neighborhood risk.

I work in GIS for utilities and know where about a mile of my feeder comes from but I can't figure out where exactly it goes after that.

Somebody probably overlaid a few maps (weather forecast, estimated fire risk, electric lines) and picked feeders based on that.

That wasn't my assumption, for what it's worth. I understand that it's about the lines not the end points. It's a good point though, that the lines they worried about the risk of might be nowhere nearby.

I suspect you're right about what the process was. I think a more succinct version of my view is this: I think we shouldn't need to speculate about this, we should have the details of this process available to us. I understand that info about specific lines might be sensitive, but we should know what the process is for this.

Something that might not be obvious: the shape of feeders can change based on switching operations. New PV on the big expensive house next to open space can create problems (you now have a generator right at the end of the grid) and new EVs can be roughly as much demand as a new house. Sometimes the fix is to change what feeds what, especially in a dense city. The utility obviously knows what feeder every meter is on but may not have anticipated (or paid for!) bulk calling the exact customers based on that. CA had rolling outages (caused by Enron) and implemented a system for notification but it was work, took years to design and put in place.

The systems for managing switching and restoring power are quick/live/critical whereas the systems for building / maintaining are more slow / archival / non-critical. Both are typically legacy systems mired in an interconnected web with other legacy systems and very, very slow to change. Anything that looks like a bulk customer / phone list api is even slower.

The Practical Engineering Youtube channel has a very interesting video on black starts of an electrical grid [0]. Are there any additional challenges in turning a grid off?

[0] https://www.youtube.com/watch?v=uOSnQM1Zu4w

There's a pretty big challenge, which is that you aren't turning off a grid at all -just particular transmission lines. Your typical power flows rely on those lines being up, and without them strange conditions can occur. Its sometimes just as bad to have an unusually lightly loaded line as it is to have an overloaded line. Lightly loaded lines with lots of generation end up messing up line voltages due to capacitive loading. You can end up having to shut down a line or taking other remedial action like shutting down generation because not enough power is flowing through a line.

Ehm... Sorry, no. To prevent wildfires from the grid you need to make most cable underground, where you can't keep them in a clear area, large enough to avoid lighting up a fire if a pole drop.

Why is it that the costs "trickle down to the electricity consumers" while utilities like PG&E report record profits? Are profits more important than safety, or customers?

I suspect we'll see more openness to this with homes in the future coming with batteries to power for 12-24 hours without the grid.

In my experience 12-24 hours won't help you enough in a city, and is woefully inadequate in rural areas.

There are better ways to spend your money to cover risks than batteries. Maybe if you were getting a battery for other reasons, or as a luxury if you are wealthy enough.

I think if you have a big enough time of use differential, and you have significant use during peak pricing, you can justify a local battery system for peak shaving and get backup power 'for free' or vice versa.

12-24 hours would have outlasted any outage I've experienced, until I moved to my current place, but it has a propane fueled whole house generator (and low utility rates with no time of use component; i haven't attempted the math, but I'm pretty sure running from my generator is way more expensive than utility power... OTOH, I've had many utility failures and only one generator failure (12v battery failed due to charger failure))

With a battery, you could run your generator at peak efficiency when it's running, and idle it otherwise, significantly reducing your running costs.

Yeah, but even at peak efficiency, I don't think $/kWh on propane is going to be better than $/kWh from my utility. A battery system would be nice though, all things considered, so I could run the generator for fewer hours during an outage: only running the generator when my demand is high or the battery needs to charge would reduce the time spent at low load where efficiency is poor.

> so I could run the generator for fewer hours during an outage: only running the generator when my demand is high or the battery needs to charge would reduce the time spent at low load where efficiency is poor.

There is little point worrying about marginal efficiency when power is out because saving a few dollars over a few hours is not worth optimising for. - High fixed costs (batteries & inverter) would dominate small operating costs (propane, wear&tear).

Marginal efficiency could allow for longer system runtime with the stored fuel, though. That would be important if fuel deliveries were unavailable, and since utility outages (for me) correlate with poor weather that causes poor road conditions, it seems like a reasonable thing to consider. Also, I think maybe a battery supplied inverter would give me less wobbly power than I get at low load --- I'm pretty sensitive to flickery lighting.

Probably not worth it though at the moment. I suspect it would be simpler and lower cost to double my fuel storage capacity. Lived experience says in 6 years, the maximum duration outage was less than 3 days, and my storage capacity is plenty good for that.

From your past comments I think you are near Seattle, and that area looks fairly red on the USGS national seismic hazard map!

> the maximum duration outage was less than 3 days

Past performance is no guarantee of future results!

My semi-rural parents were at one end of the faultline of the Canterbury earthquake and my sister lives on a rural property on top of the Kaikoura earthquake.

If you are in an earthquake zone, my lived experience has been that planning for more than a weeks disruption is very worthwhile.

They both have wood burners and had months of firewood so heating was fine. They used gas cookers for a while. Both have septic tanks and both had non-powered water supplies (no electric powered wells - they had tanks fed from roof and hillside). I think my sister lost a couple of chest freezers of meat because power was out. Many people had no serviced power for an extended period of time. My sister lost their ADSL internet (their property has zero cellphone coverage). She'd be okay now with her Starlink.

After an earthquake, you cannot buy a generator (they are all sold quickly).

Getting gas deliveries might be difficult (bridges and roads were fucked near my sisters).

I will instal solar with batteries and I live in a city. Power, sewage and water were out in many parts of Christchurch for a long time after the Christchurch earthquake. I'm no prepper. I am just happy to hedge my insurance bets if the price is right!

Then again: after three major earthquakes I would hope we all learn a little...

If you're visiting NZ then come visit.

Nearby site generator? Diesel or Natural Gas, depending on reliability during power outages?

Depends what you need to do with that. Sure, you can't 'live normally' on battery power for too long, but if you prioritise things, your battery can last for a much longer time. Fridge/freezer to prevent spoilage, phone/laptop charger for comms, a couple of lights, a microwave oven. Heating/cooling is tricky as that's a _big_ draw, but there are ways to be efficient with that too.

> phone/laptop charger for comms

You may not need to do that for long, depending where you live. My wired ISP does not provide service if there's no utility power at their box down the street. The cell towers provide service for about 2-4 hours. And after that, I've got inbound only comms (radio / tv).

> Maybe if you were getting a battery for other reasons, or as a luxury if you are wealthy enough.

Batteries are cheap. A 5 kWh battery storage pack can be had for ~1500€ these days, that's enough to ride at least the fridge, freezer and LED lighting for a day or two through a major event. In rural areas, add a small 1-2 kW generator and a transfer switch for another few hundred euros, and now you're completely set even for larger outages.

If you're buying a generator, why bother spending 1500€ more? Plus you are ignoring the cost of a charger and inverter.

I found a cheap 5kWh wall unit includes everything for $4,350.00 NZD (things are more expensive at our edge of the world). Median after tax pay in New Zealand is a bit over $52,000 per year. I don't consider nearly a months earnings to be "cheap". Maybe you are just wealthy?

> If you're buying a generator, why bother spending 1500€ more?

You can use a battery pack not just for emergency storage but also to store energy in cheap times (i.e. at noon) and draw from it at expensive rate times. Of course that requires a variable-rate tariff, but these are getting rolled out even in Europe.

> Plus you are ignoring the cost of a charger and inverter.

Indeed, you're right (my bad, mis-read an offer), but even then - a full system, including battery pack, controller and an inverter that can additionally be fed from a solar panel is just 2.500€.

> I don't consider nearly a months earnings to be "cheap". Maybe you are just wealthy?

Distribute the 2500€ across 20 years of lifetime and it's 10€ a month, and that's before the potential for savings you can get by attaching a solar panel. Yes, a (relatively) large upfront investment if you are in remote places like NZ where everything is expensive, but in Europe or the US? Almost a no-brainer, assuming you have a place to install it.

[1] https://www.fm-solar.de/batteriespeicher/batteriespeicher-pv...

We already have those batteries: Electric Vehicles. We just need some kind of vehicle-to-grid standardization.

What happens when the power goes out when your car is not charged, or if you want to drive somewhere during a power outage?

There is a bad snowstorm coming in folks. Snow, high winds and temperatures best thought about in Kelvins. Just joking with ya-all. Looks like we will be snowed in for a week or so. To keep ya-all safe, the utility will be turning off the grid. Remember to stay warm.

That's actually a real concern for using this strategy in Colorado in the fall. The Marshall Fire (mentioned in the article) was followed by snow and temperatures well below freezing. Sadly, this probably killed some pets that escaped the fire.

Reply on news.ycombinator.com