Hacker News Reader

Comment by baybal2 | Hacker News Reader

Wait a second, and think how real is the chance of protection relays having an Internet connection.

Directly? Low.

Indirectly through serial concentrators and RTUs and Scada access, almost 100%.

The FERC / NERC security standards that fell out of this event are largely a bad joke, but had the benefit of at least requiring North American utilities to do something at all.

Any transmission operator of size and most generation operations effectively require real time Scada for anything of meaningful capacity and they're sure not paying someone to sit next to it reading numbers off over a phone. They may not be IP addressed individually but to put it in perspective, I had a communication engineer tell me with a straight face that the fiber mux I telneted into over their local wifi wasn't internet connected, and thus the protection relays on the 500 kv substation the next building over were serial only -- over said fiber cables.

As an architectural building block we have been stuck with Firewalls, IDS etc. We need a new kind of isolation device – an air-gapped one-way data transmission gate – that can sit between critical safety control systems and their monitoring systems. These read-only monitoring systems can be connected to networks (presumably reachable to hackers through some means). As long as the safety control systems cannot be modified by any means, these types of failures should not happen.

Also, irrespective of software safety, for a system like this grid power generator why can't they design physical safety relays/cutoffs? If the generator sped up beyond it's tolerances, it could cut-off. Why should it generate shocky torque that can damage the rubber grommets? That could be prevented by design?

"We need a new kind of isolation device – an air-gapped one-way data transmission gate – that can sit between critical safety control systems and their monitoring systems."

Those are called "data diodes". They involve optical isolation. True one-way is very limiting, though.

A bad idea: the "bidirectional data diode".[1]

[1] https://www.fibersystem.com/wp-content/uploads/FS19189-Datas...

One-directional data transfer isn't really a problem: just have a wire that only sends data in one direction. If you want to be extra paranoid, make it an optical link. Or wireless transmit-only if that fits your application. Downside is you need more bandwidth since you can't have the receiver ask for a specific subset of data.

The trouble always starts when someone wants remote inputs for some reason. Then you really need to start to think about a) how to protect the gateway systems and b) how to make sure that even if the gateway is cracked the damage is limited.

> If the generator sped up beyond it's tolerances, it could cut-off. Why should it generate shocky torque that can damage the rubber grommets?

This was not the generator speeding up beyond its tolerances. From what I understood from this article, it sped up slightly (still safely within its tolerances), but the difference in frequency made it out-of-phase with the power grid. Once it was connected, the misaligned electromagnetic field instantly tried to align the shaft (and match its rotation speed); that's where the sudden torque came from.

(This Wikipedia article has a description of the procedure using an older instrument: https://en.wikipedia.org/wiki/Synchroscope#Process_of_Synchr...)

The generator wasn't sped up beyond tolerances, the safety system that kept it synchronized with the wider grid deliberately brought it in out sync, like ramming a car engine into a much lower gear at high speed. The force has to go somewhere, and from the perspective of the generator the mass of the grid might as well be infinite.

As for better architectures, we're working on that, but industry is lethargic, lazy, and often organizationally dumb.

But there are limits to isolation. AC grid frequency and voltage and load flows are changing constantly, if normally by small degrees, and propagate at effectively the speed of light. It's impossible to have purely static immutable protection schemes for all but the most trivial of equipment.

Not to say it can't be a lot better.

> why can't they design physical safety relays/cutoffs?

They can and they do. Or at least "they" used to. Nowadays such safety devices are very often software enabled. Because it's cheap and also because it checks a box on the marketing brochure that every customer wants: "Software upgradability for lower maintenance cost and reduced down time." Oh the irony.

I would imagine the chance is higher than you think.

Being able to remotely monitor and manage such systems may lead to big cost-savings, if it enables the same number of employees to monitor multiple locations.

Seeing as how many tech companies routinely cut corners on security, one could easily imagine a non-tech company doing the same in an attempt to maximize profits.

It's not just remote monitoring. Protective relaying involves some disconnects sending signals to distant disconnects to tell them not to disconnect. The idea is that when you have a sudden overload like a line down, you want to trip the breaker closest to the trouble, and not others elsewhere upstream. Without this, small disruptions take out too much of the grid. See NYC blackout of 1965.[1]

This is why large transmission networks require some capability for fast transmission of data between circuit breakers. Not much data. But it's so much cheaper to send it over the Internet than to have separate microwave repeaters or separate cables or gear to send data over high-tension power lines.

[1] https://en.wikipedia.org/wiki/Northeast_blackout_of_1965

Reply on news.ycombinator.com