Volunteer DEF CON hackers dive into America's leaky water infrastructure
theregister.com
theregister.com
Before anyone starts rambling on about politics too much on any side, i blame all sides. What i recall from my basic education so many decades ago is that government should help to provide at least some fundamental areas of infrastructure (e.g. roads and such, etc.), and then commerce (private enterprise, etc.) can take place above it, and than things proceed from there, yada yada.
I don't know what is more basic infrastructure than water and its associated management? Where is the NSA in all of this? What about Department of Homeland Security's Cybersecurity and Infrastructure Security Agency (CISA), where are they in all of this?
Clearly, this story ticked me off, and apologies for that...but, if we're at the stage in society where volunteers is a viable solution for a very fundamental element in life...then clearly lots of other things have severely failed.
Properly fixing infrastructure has become expensive as construction costs have skyrocketed in the last few decades.
Look at the CHIPS bill, huge infrastructure gain but also was a part of the 9% inflation because building this infrastructure is expensive. The American people were warned about it causing inflation but wanted it anyways. When inflation hit they all conveniently forgot the cost of new infrastructure in the US and chose to cling to lies about how disastrous the last 4 years have been.
That is why today no one does anything about problems until a bridge collapses. You can make more money today by passing the buck to the next generation and passing blame to whoever came before you.
> but also was a part of the 9% inflation
Yes there are other reasons, COVID payments being another reason, which citizens were warned about. The first payment was signed by Donald Trump himself. Any time there is an increase in spending it will cause inflation. Increasing defense spending is another cause which happens yearly.
The CISA does this type of thing all the time. Here is an example just from their recent news releases in the past week: https://www.cisa.gov/news-events/alerts/2024/11/21/cisa-rele...
The CISA offers to assist infrastructure providers in a number of ways: https://www.cisa.gov/resources-tools/services/assist-visits Presumably the NRWA could have asked the CISA for help with this initiative.
The CISA has 3,000 employees and a budget of $3B, which is a lot, but not enough where they can be involved in everything. For reference there are 12,500 utility scale power plants and 148,000 public water systems in the United States. The scale of the problem means they must be mostly an advisory organization, where most of the work gets done by people at the infrastructure organizations or, sure, volunteers.
Now, I’m not saying that the CISA is doing a good job, I genuinely have no idea. Determining that would take a lot of knowledge, probably insider knowledge, and weighing what they’re doing against what resources they have available. But them not being involved in some random project that showed up in your newsfeed doesn’t mean they have severely failed.
> ...But them not being involved in some random project that showed up in your newsfeed doesn’t mean they have severely failed.
I surely and honestly hope that you are correct!
If you want to make a secure military base, you do not hire a spec ops team to develop one. If you want to make a bulletproof vest, you do not hire a gunsmith to design new synthetic fibers.
Having offensive teams on hand to verify and validate is necessary, but largely orthogonal to the task of design and development. The skillsets are highly dissimilar.
The fact that people think this is the golden way shows how absolutely intellectually bankrupt the entire commercial cybersecurity industry is on a theoretical level. And the complete inability to protect against the regular and standard threat actors today shows and supports that empirically.
When you can be attacked by groups of 20s something with only a computer and fulltime to attack you via your smartwatch or social networks you would rethink about cybersecurity. Your example is linked to physical but not virtual spaces.
Furthermore, groups of kids do not constitute a technically sophisticated threat. If you are struggling with them, then your security is grossly inadequate against commonplace and routine threat actors today like individual opportunistic criminals. Which are themselves grossly inferior to commercially motivated cybercrime groups. Which are themselves grossly inferior to state actors who are also commonplace and routine threat actors against national infrastructure.
The fact that commercial IT cybersecurity companies continue to fall prey to such bottom-feeder predators should tell you everything you need to know about the value of their recommendations and "expertise". If you disagree, present a software system developed using "offensive security" focused methodology (which if you present no definition I will define as using offensive specialists to design the security of a software system) that has been explicitly audited to be secure against a small team (let us say 3 person-years) of offensive specialists. And no, the absence of evidence: "There have been no publicly announced hacks on this system" does not constitute the evidence of absence. Only explicit audits that actively test the security of a system provide evidence of absence.
And, just to head-off any claims that is an impossible standard that nobody reaches, I will point to Common Criteria SKPP certifications and the seL4 proofs of correctness as examples of the alternative that has reached such standards.
It's hard to imagine significant help the government could give us, short of $100K+ / year to hire a security engineer. Even then, how would small utility companies find / evaluate them? We already feel saddled with how many hours of mandatory training we have to go through each year (e.g., board members have to sit through training to remind us not to direct company funds into our own bank accounts - I doubt that's a training issue!). Looking at our two neighboring water companies, their setups have very little in common with ours or each others. So any training would be too generic to be of much use.
I have to give credit to our company's chairman / general manager (himself 84-years old). He works to have diverse expertise on the board. He's a retired wild-land firefighter, plus there's a retired bookkeeper, a retired state employee, a retired farmer, and me. As a group, we have a lot of experience in a lot of areas. I suspect most small, rural water companies haven't found a way to have that breadth of knowledge available. But, even then, there's only so much that two full-time plus two half-time employees can do. And it's hard to have more while keeping rates sane.
On the security side, I'm no expert, but I pay attention. I do my best to help the employees understand the dangers of phishing and of downloading things onto company computers. We have a consumer grade router and none of them know how to get into it to open ports, etc. Even if things were locked down hard, we couldn't afford to hire someone to maintain that state as threats evolve. Our total compensation budget is $150K / year for all employees, and 80% of that is needed for state-licensed water treatment operators.
For us, all of that means being resigned to the fact that hacks will happen. It's more about minimizing the damage and being able to recover. I've pushed for using cloud services for things like customer billing - these services are not the ultimate in security, but they're far better than what we could do with software running on our computers. I've also put append only / offsite / offline backups in place so we can recover from encrypting ransomware (which has hit us before, luckily we could just wipe and re-install / restore).
The biggest thing we've done is around SCADA (software / hardware involved in our water treatment and distribution). When we put SCADA in place ten years ago, I pushed for it to be read-only. That is, it can be used for monitoring and alerting, but not for controlling anything (there's literally no hardware in place to do so). So, hackers can see how much water, chemicals, etc. we're using, but they can't directly change or shutoff the water. They can make us think we need twice the chemicals, but that will be a red flag for the operators who have to manually implement it. Even then, we do manual daily testing of our water in our lab and have monthly testing done by state-licensed independent labs.
But, when we put SCADA in place, I had to stop and think a lot about this given that I wasn't a security expert. And, of course, the vendor was certain it was 100% secure (a red flag in itself). But at least I had experience thinking about software issues / impact. The operators certainly wanted automated control, as that would save them from having to drive to the plant in the middle of the night. And without automated control, we have to use more chemicals, as we can't optimize usage by reacting in real time to changes in pollutants in our source water. But we'd all rather deal with those downsides than find out someone has compromised our water. So it was easy to sway them.
It's far from perfect, but we've mostly limited potential damage to things that aren't deadly, literally or figuratively. Without someone like our chairman / general manager being dedicated to bringing in diverse expertise via the board, I don't see what chance we'd have. And in certain parts of the country, it would be hard to find security (or at least software) expertise to sit on the board.
Something our state is doing to attack the lack of economies of scale (in this and other areas) is trying to force rural water systems to merge. About half in our area have been folded into the water company for the "big" city (5K people), 20-miles away from us. That's too far to move treated water given our local terrain, so they have to keep running the systems from afar. But at least they can spread (e.g., IT) expertise across these systems. Where this has happened near us, water rates have roughly doubled. Maybe that trade-off is okay, at least for those who can afford it?
"Have to"? As in 'obligatory?
The law was in reaction to a suburban water company board in our state paying each of its members hundreds of thousands of dollars per year until it went bankrupt. That comp is not crazy in massive tech companies, but it is in small water companies. On our board, our compensation is up to $100 off our water bill each month and one bag of trash in the company dumpster each week, if there's room. That's in return for a ~2-hour meeting each month and 2~20 hours of work on various initiatives / committees (or training) each month.
While i understand that funding might be very limited or non-existent, actually, your situation sounds almost exactly the kind that could benefit from support from a government entity. It could run the gamut from free training, to actual technical advisement, support, to direct funding for cybersecurity/protection operations, etc. Tlo be clear, i don't mean that the gov. comes in there and takes over - no, no, no. I mean, that some gov. entity comes to you asks what they can do to help. Sure they can advise on best practice, but its your show, and you should run it your way....its simply that they might have the know-whow and funding to help you and your water company. At least, that's what i figure shoiuld happen to help communities like yours and others! :-)
Only one side is consistently obstructionist toward progress.