Now just wait to see what will happen to your insurance rate after you pay the third ransom.
They certainly will begin to understand the need for backups.
Another issue with backups, is are you restoring to an already infected / immediately infectable state?
I think the better closer is “The certainly will begin to take security, training, and best practices seriously”.
It sucks locking things down for each employee, and subjecting them to bureaucracy to unlock things they need to do, but it's better than ransomware.
It's unrealistic to expect every employee to catch hacking attempts 100% of the time.
It's like compartmentalization on a battleship. A single hole won't sink it, in fact, many holes won't.
Good post. I don’t mean this criticism for you specifically. But, why is there an assumption among HN types that there are no bad-actors among the insiders? You can have all the safeguards you want, but if an insider deliberately installs something, you’re screwed.
In some industries — armored trucks, banks, military stuff — there is a huge emphasis on background checks, security clearances, and the like to weed out bad actors. (And, even then, it often fails.)
I sense there is nothing similar for employees handling the company’s data. Obviously, there might be background checks and the like — hell, McDonalds has background checks. But, I’m not aware of the intensive FBI-style screening you see in the aforementioned realms.
Am I wrong?
How many thousands of people, for instance, could corrupt or lock the data at, say, Amazon? Are these people scrutinized to the same level as standard Brinks Armored Truck driver? I doubt it.
- is protecting against internal sabotage actually different that protecting against external attack. I don't think it's all that different. It comes down to authenticating actions and enforcing the principle of least privilege. If you built a system that was actually secure (i.e. one that depends on reasonable inconveniences, rather than one that depends on people to be perfect all the time or is so inconvenient it inclines them to do the digital equivalent of jamming the door open) it is likely that it will be secure enough against most internal saboteurs.
- is protecting against internal sabotage going to pay off? Most people probably aren't inclined to deliberately target their own company. It's far more likely that there is a bad actor in the world who wants to target your company, than that there is in your company. And making a person's job secure less stable is probably going to make them more likely to be a saboteur, so you should carefully evaluate whether gratuitously adding stress to someone who might get behind on their mortgage is a good idea. (Which I suppose is what this kind of background check would cause.)
“Most” people are law abiding. So, I agree with the first sentence.
The second sentence, however, has little support. The universe of people who can do these types of attacks is large, but not universal. You need computer skills. Necessarily. Those with computer skills are usually already part of the industry. How many disgruntled people pass through the FAANGs each year? Now add in all the IT positions at the banks.
In short, there are probably tens of thousands of domestic ‘bad actors’ who have (or will have in their careers) access.
Probably more.
Your argumentation doesn't challenge the truth of falsity of the statement, nor does it go towards challenging my conclusion. It seems to assume I've said "there are probably no internal bad actors", when I've said "securing your system against external bad actors will deal with the cases more likely to occur, and will usually be sufficient against the less likely cases".
'X is more likely than Y and preventing X mitigates Y to a tolerable level' is not equal to saying 'Y probably doesn't happen'.
> How many disgruntled people pass through the FAANGs each year? Now add in all the IT positions at the banks.
Obviously the probability increases significantly with the number of employees, but I don't think that switches the probabilities. Still, the most important companies to consider are the companies which, unlike FAANGs, aren't really in a position to make independent judgement about their risk profile, but whose existence depends on their records/data. And even someone with as many employees as the US government probably has more people outside of their employ who want to target them than inside, although surely they always have some of both.
I'm not sure what the relevance of your reference to computer skills is. In order to ransomware a company, as far as I know, you need to buy software off the darkweb and run it from a vulnerable location. I suppose technically that requires computer skills, but it's surely not what you mean. And the companies which are vulnerable to ransomware are not all employers of significant numbers of technically skilled people.
In any case, I don't think even a gratuitous reevaluation of the probabilities significantly changes my conclusion. Even if we assumed the improbable notion that every single company in the word has a disgruntled employee and that there are no external crooks, the process of securing the system against external crooks will make it far more survivable against single internal bad actors, and the effect it has on the employees will be less likely to produce internal bad actors.
The entire issue just leaves me with a nagging feeling that something fundamental is being overlooked. There is something profoundly different in modern companies that didn’t exist a few years earlier; namely, a very tight concentration of data/ops/control. Whether it’s external or internal, it seems a single person can do catastrophic damage to the company AND the customers. Fifty years ago, for instance, in a typical auto plant, I doubt a single person could have truly devastated the company (short of a bomb or arson or something). Nowadays, we’re moving towards systems where not only could a single bad actor cripple the company but also cause all the cars already sold to stop. (I exaggerate a bit, but you get my point I hope... )
But in the corporate world, theres gotta be huge variance, but so many don't give a flying flamingo who's scoping out what, unless somebody is forcing the issue (and also auditing and reporting to the compliance department, whatever thats for).
They know the people in the NOC/SOC, the C-suite has equity, there may be physical access control, cameras and proxcards out the wazoo, but when Marge from bizdev needs those emails for the marketing newsletter or whatever, she is gonna get them immediately and hand them right over to the intern or vendor or Doug, whatever his job is.
For all the obscene value that the data and access represents, its encrypted, right? What could go wrong? Want to background check the sales people? But... look at this guy's resume! He's only asking 80% of the market rate! These dialysis machines sure won't renew their support contract by themselves.
Best case scenario is that the costs mount even higher into the stratosphere and people start demanding a second look. It's been a while, Maersk, JP Morgan, TransUnion, Colonial Pipeline, Beef, Hospitals, Schools, the OPM (for god's sake...) billions or trillions of dollars. It doesn't seem to be a priority.
No. It never takes only one employ clicking a bad link. It takes that click, plus a browser/email/os system that allow for random code to executed. It take an IT department that has allowed individual non-IT employees to use computers with elevated privileges. It requires a management structure that has failed to invest in proper off-site/cold backups. It requires an organization that doesn't have a proper business continuity plan.
And at the top of the incompetency pyramid, it requires a vendor that sells an email system that allows evil email messages to somehow infect entire operating systems. Want your email to connect to your office suite? Sure. Want to install random software based on clicked links? Sure thing. Want to update your firewall, install a new browsers and simultaneously backup all your encryption keys to a random server in the far east? Why not! Anything to make your operating system experience seamless.
Locking down admin access is less about protecting the local machine and more about preventing the laptop from becoming a jumping off point into more important pieces of infrastructure. It's by no means fool proof, but it adds another security feature that has to be countered.
I don't know whether it's worth the trouble it creates, but I don't think it's totally without justification.
- A phishing email which can pwn you without user interaction is basically unheard of.
- Even malicious sites generally can't do anything bad simply by visiting them. (and yes, I'm aware browser exploitation exists, but it is exceedingly rare)
- Ultimately, it's entering your credentials in a malicious site which is what puts users at risk. A user must click a malicious link (sometimes two) and then intentionally enter their credentials into the malicious site.
Between this, and the fact that users must read emails, visit sites, and enter their credentials over and over, just to get through their workday, I believe the outcome is that user education doesn't amount to much. It would be much better if a normal user's workflow didn't usually require clicking on email links and then entering their credentials. The fact that this is required means that even a savvy users will eventually be tired / rushed / working on automatic and get owned.
If you extend from "email" to the other communication tools that companies use today (and do use for inter-company communications too), there actually have been a number of these in the past year.
Outlook [3] had one that didn't require downloading the file, exactly - the "Preview" window from just clicking the attachment once, was enough.
Microsoft Teams [0], Jabber [1] and Slack [2] were all hit by real 0-interaction RCEs.
[0] https://github.com/oskarsve/ms-teams-rce/blob/main/README.md
[1] https://nvd.nist.gov/vuln/detail/CVE-2020-3495
Why? Secretary gets a call from a nigerian prince, starts that letter.exe she gets in her e-mail, her computer gets fscked, IT takes her drive, restores a clean image, and she gets back to work.
If the only copy of some important document is on his/her pc, or that pc can overwrite/delete the only copy, then they've fscked up by design... and yes, now better backups would help.
Thank goodness I didn't have access to a script that would lock up at least two of my past employers when coming up years ago? Then again, I personally haven't been that mad, but boy do I know employees who were.
I could say that we are all choir boys, but you piss on an employee, especially during a recession, well let's just say I have seen unpstanding guys rub magnets over hard drives over pure apathy. (The guy didn't know about strength of magnents, and it did not hurt anything.)
Plugging in a usb, or downloading a suspicious email is something I can see happening, especially to "those" companies.
I imagine Xfinity employees dream about it?
Open up a spinning rust hard drive and you will find two very strong magnets inside, positioned opposite each other.
We got bought. Big corp enforced Endpoint Management and a whole barrage of corporate spyware.
I am not an admin anymore. I can't even use an AdBlock solution anymore.
And guess what. I don't give a damn anymore. If the device enforces an update, so he it. If I have to double approve every external mail address when sending, so be it.
But I don't feel ownership or responsibility anymore. Should corporate overlords care. I am out.
Guess what? The big-corp IT managed computer which I only use to check email and edit Word docs is almost unusably slow, weighed down as it is with antivirus, surveillance software, centralized updates, etc. The project machine which I manage and have admin access to remains snappy despite its workload (Visual Studio) and older hardware.
Also, you gotta love the "cost savings" of let's outsource our IT, not let developers be admins on their own machine: now we have to buy secondary machines that developers ARE admin of, so they can get their work done.
And I'd argue this isn't only true for kernel development. In some cases, sure, but certainly not enough to make such a blanket statement.
Though I will agree there are exceptions, you are not one
I kept my machine as it was, explaining that if anything happened to those units (bad updates, blah blah), mine would be unaffected and mine was completely necessary.
Lo, and behold! That very day everyone was complaining how slow their computers were, how even basic websites now took ages to load, and they did.
The owner refused to admit he made a bad decision and stayed with that 'IT' 'company' for over a year, and didn't get rid of them until I'd left and no one was available who could triage, and they saw just how little that 'company' did, and just how much I was made to cover for them
That last part leaps out at me as particularly interesting: highlighting behind-the-scenes firefighting work is always tricky. Management doesn't want to acknowledge that it's necessary, while engineering maybe shies away from managerial caricaturization of what ultimately amounts to implementational minutiae. How'd you end up conveying the behind-the-scenes work you did in these kinds of situations?
And the accounting folks will not be fans of anything that costs money. They will just say "But we haven't been attacked a second time, why should we pay for mitigation services and implementations??"
Organizations can and do take many decisions of which "the accounting folks" are not fans of, the accounting people can and do say such things about the costs, but they don't have a veto. Arguments about cost of mitigation are valid in general, but leaders and owners can choose the priorities, and the responsibility and blame for these choices is fully on them (for their will or lack of will), not on "the accounting folks" arguments.
Also, sometimes that accounting argument is entirely valid. For example, look at the recent case of First American Financial - https://krebsonsecurity.com/2021/06/first-american-financial... - if the consequence of leaking the sensitive financial documents of millions of customers is just 500k, then it definitely is cheaper to just accept the hacks and pay the compensation, because investing in proper security would be much more expensive than that.
They used to (and probably still) do this. But more recently these folk are paying access brokers. A bit like bank robbers teaming up with criminal locksmiths.
> It makes sense to me that’ll happen again and I’m not sure I can say the solution is better backups.
You'll ideally need:
- Better security awareness training to cover human weakness such as spotting dodgy email and what to do if you click a link
- Patch vulnerabilities quickly as this will reduce risk considerably
- Company wide tested and sufficient backup strategy (most companies fail on this) to protect key identified data assets
- Regularly pen-test both your internal and external environments
Obviously it doesn't stop there, but those are key.
Snapshots aren't backups.
Backups that aren't physically-isolated, typically offsite, aren't backups.
1. Test and encrypt backups.
2. Don't get hacked. Defense-in-depth philosophy and rigorous, routine social-engineering training/testing. If you get hacked, it's usually game over. Defend systems like the business depends on it because it does.
3. Limit exfil: extra security for PII, exfil detection, and [HN]I[DP]S.
What OP tried to demonstrate that backups need to protect from bad changes, on physical, logical, and business layer, from data corruptions to 'oops' scenarios (i.e. drop table). Standard snapshots for sure don't protect from all those.
Also for a good backup strategy, you need - "a full start" once in a while, because corruption in "full backup" will invalid all incremental snapshots - regular restore of a backup for e2e validation
I suspect you mean filesystem snapshots and as long as the snapshot lives on the same physical media you are correct. But when you take the snapshot and transfer it to a physically separate location where it cannot be altered it sounds like a backup to me.
Vaulted offsite on at-rest storage media is the only valid way to store backups. Every other "convenient" "backup" service or snapshot replication process is a liability businesses must avoid. Lose all your data, 50%+ you're out of business, and The End.
Relevant analysis here:
http://www.vendormanagementoffice.net/2021/06/cyber-insuranc...