Hacker News Reader

Comment by ryanblakeley | Hacker News Reader

I interviewed a guy from the NOAA to ask about that. The gist is 1) the military has hardened satellites 2) they have a phone-tree system for alerting grid operators and 3) many high voltage transformers have back-up systems in hardened storage.

[1] https://ryanblakeley.net/p/solar-flare

View on HN

I won't comment on 1 and 2 but 3) is laughably untrue. I guess the word 'many' might be stretched to mean 'a few' but I've worked at more than one very large utility directly as an engineer responsible for design of large generation and transmission interface systems and these transformers are not only not spare, they're almost impossible to get quickly even at current market demand. If you toasted even 10 percent of the large generation and transmission transformers in North America it would be years before the grid would even come close to working again, and that's if there was nation state level resources applied.

These are specialized machines that require specialized labor and specialized materials from a specialized international supply chain to construct. I'd make the bet that total recovery would be decades.

This keeps coming up and I keep wondering how in the world are large transformers that complicated to build. And I don't mean that in a dismissive way but a "boy do I have a gap in my knowledge of the world" way :-)

I think that is a pretty common reaction when it comes to physical/mechanical objects. Another way of thinking about it which might be helpful is. Hundreds or thousands of very talented engineers have been working non-stop for 150+ years in a transformer arms race.

This has resulted in incredibly high performance designs and incredibly high requirements.

No doubt. But same could be said of, say, automobiles, or microchips, and many other things. Even larger objects like power plant turbines seem relatively frequently manufactured. Large transformers on the other hand appear to be closer to nuclear power plants in rarity/difficulty of replacement, and as a result for me they have a very high difficulty-of-replacement-to-perceived-complexity ratio.

Large transformers are built to a very specific custom specification. Each one is a custom engineered unit. A transformer is basically a sealed vessel full of volatile oil that you pass an extremely large voltage and current through. The machine has to be perfect. The smallest clearance issue, insulation problem, etc and you have a bomb. And that's just making it safe. We haven't even talked about making it effective.

Transformers are the sort of think that are not rocket science to build, but every single detail is very very important. It requires a team of people to design, construct, and validate and all these people need to have the skills from doing it many times before. These people and facilities simply don't exist in large enough numbers because they are very expensive.

There are also supply chain issues to consider - the steel used in transformer magnetic circuits for example is a very specific metallurgy and construction. During COVID it was quite difficult to find. Poor steel means poor efficiency which at the scale of these machines can mean millions of wasted mwh of power over time.

Thanks for the detail, this is very neat. I wonder if transformers like this are in an "uncanny valley" of manufacturing where there is barely not enough demand to bootstrap mass production like for example turbines. Turbines have very exact tolerances as well, but it seems that with higher unit counts orgs like GE managed to churn them out.

Perhaps the liquid piece is also making things more difficult; I'm sure a dry object is a lot easier to warehouse until needed.

I thought turbines had lead times on the order of years

12GW at 1100kVDC. [1] That's bonkers! I wonder what the spare part plan is for this beast.

[1] https://www.youtube.com/watch?v=0pGH1B863oI&ab_channel=Engin...

They are dry until delivered, because they already max out the transport vehicle's capability (often highly specialized transformer carrier train cars with like 16 axles at each end to spread the weight).

Any thoughts on how the situation compares in Western Europe? Too many disparate systems, or…?

No idea, my experience is strictly NA. I wouldn't want to speculate

I appreciate your perspective. Perhaps I should have said "few" instead of "many" and "parts" instead of "systems". But it does sound like your overall impression of backup and restorative capacity differs from what I gathered in that interview.

He did say damaged HV transformers would be a big deal and repairs would take weeks, but maybe that was an optimistic take. Talking about solar storms and grid damage, the range of possibilities is very large, depending on the storm's intensity and where it hits.

I work in power grids, and I can also confirm that number 3 is not true. If a large number of high-capacity power transformers are required, we are in trouble.

Reply on news.ycombinator.com