A mysterious grey-hat is patching people's outdated MikroTik routers
zdnet.com
zdnet.com
Have to wonder if those "outraged" users are ones who would have proactively fixed it themselves, or if they would've let their router happily continue to chug away as part of a botnet.
What's your point?
A hilarious and interesting example: https://www.gimletmedia.com/reply-all/long-distance
Additionally: see all the drama and issues that consistently occur surrounding bug bounty payments, secure disclosure of vulnerabilities, etc.
Would you be outraged if you came home one day and there was a plumber fixing your sink? “Oh hi, don’t worry about me, just fixing your sink. Let myself in, hope you don’t mind”
You didn’t even know your sink was leaky let alone called a plumber.
Without going any further in than he had to and without charging you? I'd probably be a bit weirded out but quite pleased as long as he didn't hang around!
I guess this is part of the issue... even for people who have an understanding of it, it's a nuanced topic and the analogies are widespread but often misleading, because they're analogous. I'd imagine most people don't even care so long as they can access facebook.
If you think of this like physically accessing your house, it's going to seem bad. That's probably why people got upset.
You come home one day, entering by the main door as usual, and when going down in the garage you notice the key on the floor with a message saying: "Your garage door were not locked and everybody knew about it, so I came in to take the key on the inner lock, closed it from outside and slipped the key back under the door so nobody else with bad intentions can enter anymore".
In the situation you described I would be pissed off, but not in this one, and IMHO it is closer to this case.
Which I guess is something a gray hat would be happy to see
Also, the alternative in this case might be a water leak that ruins your whole house.
I came home one day to a note on my inner garage door: "you left your door open, I closed it for you ;)". No name, nothing... just someone entered my garage, wrote a note, closed it and left.
I took a quick survey to see if any of the obvious valuable items were disturbed or missing and none were. I was more upset with myself for letting that happen then a stranger "fixing" my security vulnerability.
EDIT: and another anecdote was that my neighbor let himself into my house once when a water leak was discovered outside so the water could be shut off. Saved me thousands in potential damages that he caught it early... I can't say I'd be all that upset finding a plumber under my sink fixing something but that's just me.
First, your sink is not part of a botnet (assuming it's not a smartsink, I guess). By leaving your machine unpatched, you are causing harm to others.
This makes the ethics of this sort of grey-hat hacking much more murky IMO. I'm willing to concede that the grey-hat behaved unethically, but I also believe that leaving a machine unpatched makes the machine's owner at least somewhat responsible for how that machine is used.
Further, I do not think it's reasonable to both claim that this sort of grey-hat activity is unethical and also claim that owners of unpatched devices have absolutely zero responsibility for how their unpatched machines are used. I.e., if we condemn this grey hat (assuming he simply locked to door and left and did nothing else) then we should also condemn the owners of botnet'd devices for the way in which their negligence causes harm to others.
If others can't break in and fix your stuff when it starts effecting them, then you should be held at least partially responsible for how your stuff is used by criminals.
Second, physical presence can be a privacy intrusion on its own and without any willful intent. E.g., a grey-hat plumber who is purely altruistic might never-the-less accidentally catch a glimpse of you naked. On the other hand, cyber presence almost always requires intentional snooping to cause a privacy violation.
That's why fixing router vulnerabilities is so important: if left unfixed, botnets use them to cause harm to other people.
And in the case of an multi-tenant building, if one person's actions (or lack of action) was causing problems for other tenants, you can safely assume the landlord would let themselves in to fix it.
I also heard a story of a guy who's house burned down. The neighbor saw it very early and did nothing about it cuz not her problem. The homeowner was devastated.
So yes, if you see incredible destruction going on, it's ok to go fix it.
I cannot imagine why anyone would agree to be the first target on site. That seems like a very easy way to get killed or injured.
Different people will react differently to any help you may give them. In this case, one could possibly agree that getting these machines locked down so they longer present as a threat to others is the moral thing to do, irrespective of the legality of the action.
But that is a judgement call for the individual to make knowing that there are potential consequences for their actions.
A more appropriate one would be a stranger changing your lock for you because vagrants have been going in and out of your house without you realising.
Now doesn't that sound more appropriate, good neighbourly and helpful? What do you have to be outraged about?
If you had a problem with strangers violating your property you should have fixed it yourself before it became common knowledge in the neighbourhood that your house is easy to walk in and out of without your consent.
In ~ 2002 I was off to college with my Linux workstation. IIRC, the vulnerability was in the CUPS web UI. Someone filled the volume with a giant /tmp/YOUR_SYSTEM_IS_INSECURE_UPDATE_NOW file, and shutdown the affected service.
It could have been much worse.
Trespass to save people from themselves is one thing. Trespass to save the public is quite another.
So yes, it's actually fixed, but now you know that someone you don't know broke into your house without permission nor supervision and you don't know what he's done/seen/stolen in your house.
Is it possible that they (perhaps mistakenly) believe that communicating with you could open them up to civil liability?
I'm curious, for vulnerability-by-inaction like this, would there be a legal difference if sent emails were posted to a public blockchain?
The intent being, if you're later sued for harm caused by your compromised hardware / IoT devices, you cannot claim ignorance as easily.
End goal, of course, being that people care more about patching their devices.
That's not necessarily easy to prove, in the same way the defendant could claim emails were trapped in spam filters, etc. or more realistically, the burden of proof is on the claimant so the defendant wouldn't say anything if they're smart.
There, nothing was admitted.
I would hope any well-intentioned and reputable company would not mind, but some might not want to admit any of that! Plenty of ammo for anyone who subsequently blames you if you then fail to remedy the situation in a timely fashion.
If the company is too small to monitor their own pages then I'd expect them not to be worried about this sort of liability (ie knowing of a breach, they're too small to be sued for much, presumably: if they were bigger they'd know about it already).
Both have an abuse / phishing declaration form online. I signaled both pages, and they are still up for the moment.
But seriously, thank you for taking the time to do this.
Have browsers extensions improved for this? When I last checked 5 and 10 years ago, it didn't seem to work.
I haven't looked into it in enough depth to be 100% convinced to trust it with my financially-linked passwords. (In reality, it's almost surely good enough, but I haven't reached that informed conclusion yet.)
[0] - https://www.blog.google/products/chrome/chrome-password-mana...
1Password X for Firefox and Chrome, 1Password on Safari have pretty much solved this problem. The vast majority of passwords I fill are CMD + [shortcut].
Generating and saving new ones works like that maybe 50% of the time. The failure rate however is driven less by the extension technology and more by the password form itself.
The obvious solution is to remove all users from the internet.
You and I can tell the difference, but to the sort of people who run vulnerable servers, perhaps a legitimate email about server security looks indistinguishable from the others ("Hi I'm from Microsoft technical support. Please let me in to your computer to help you fix it").
Wow, that's a phrase I haven't heard in ages.
Most webmaster@ or admin@ e-mails aren't monitored at all, or so flooded with spam that it's easy for things to get lost.
That's not it, not all of the time anyway.
Over the summer I discovered a third party mail server with a missing DNS entry. It was like that for months and all their mail was getting flagged as spam.
I sent them an email (from an account that wasn't flagging their mail as spam) pointing it out. They fixed it within 24 hours but I never got a single reply.
Politeness essencially disappears once you can't see somebody's face. 10 minutes in any online game should be proof of concet enough.
I also think the rude behavior is a combination of both anonymity and "I'm never going to see or hear from this person again".
So I forwarded it to spoof@ebay.com with the message "reporting phishing email" or something. Somehow that report got "handled" by a clueless, non-technical, front line rep who thought I thought the email was real and was inquiring about the contents of the email. Pissed me off that the email wasn't handled by the correct department. I won't be bothering to forward any more phishing emails anymore.
How do you get that authority to do that? What does "shut down" entail? Does that mean you can unregister or hijack domains? I'd like to know more about this, as well as the accountability process and where I can report abusive behavior that will actually get addressed.
No reason to ask at all honesty, it's just been one of those curios that pops up in mind occasionally
Typically a registry will just fwd it on to the registrar. That said, registrars tend to take complaints from the registry more seriously imo.
I usually only notify .edu or nonprofit organizations and completely ignore large corporations. Sending an email to a larger organization usually gets lost and nothing comes of it.
They view the message as showing up a failure on their part and they do not want anything showing that they have made a mistake in some way. So, they do not acknowledge your message as it provide a means of tracking that failure.
For those cases where it is not fixed, there is no-one who cares to do so.
In the past, I have made communications with website admins about various aspects of their sites (non-security related) when they poorly relate to those of us who are getting older and have increasing eyesight difficulties. The usual response has been "No one else has complained, so take a long jump off a short pier - our site is perfect." I sometimes try to explain that people won't continue to visit if the experience is bad, nor will they bother highlighting that there are problems. They will generally still respond with "shut-up and go away."
You just leave them to their ineffective site and move on. Very occasionally, you get back a thanks and see improvements made, but that is rare.
I can't be bothered to report it to them.
Out of curiosity, do you receive answers at all?
If not, there could be a technical reason rather than the decline of human decency: your including the link to the phishing page gets the message filtered away by automated security software.
* their IT person I think was really just the person who was best with computers.
This could very well be what's causing the outrage from operators... suddenly losing connection with your router that's in some data center 3 hours away - requiring a drive-over just to discover it's some dude adding rules to your production equipment would be upsetting.
There's legitimate reasons for remote operators to have remote access from outside the network. Obviously the router should be secured with latest updates that guard against known exploits, but this could be a major pain for some operators.
(you'd also have to roll back to some backup since there's no telling what else the guy changed, even if you feel he's more-or-less trustworthy... which means more downtime for your customers)
https://pikabu.ru/story/vzlom_routeros_5924128
assuming, at least, that the Google translation is decent: "It seems to be even good, but the admin’s account has severely cut the rights, the attackers created another one with full rights. The office is far away, the provider settings are pppoe, we can’t remove back-ups, we can also unload the config, advise how to be?"
Would you rather leave the hold open and be happy in your ignorance if the security problem in your network?
That also makes said system useless for getting work done.
Many people do not care about security at all; they just care that it "works". If we want the world to be more secure, the best (but hardest) way to make that happen is to make it cheaper/easier/faster to be secure than to be open (for example, Let's Encrypt).
I told the same company that the certificate had expired in one of their sub-domains. It intrigued me that the first 3 of their tech team didn't know what that meant.
Still they never said thank you.
I logged in and reset the password to gibberish and emailed them to let them know what had happened, assuming user error (email was a firstlast@domain, so relatively easy to mess up I guess)
A couple of days later I received an email from the company asking for my photo ID. I politely said I wouldn't feel comfortable providing that and advised they get email confirmation from users.
I didn't hear back for a couple of weeks and thought nothing more of it. Then a notification that 'my' payment to a fast food place had bounced (or been charged back, it was hard to work out tbh). I figured I'd ignore it because extradition to the states over $36 seemed unlikely.
A few days later I get another mail from them replying to my earlier mail about email confirmation and not mentioning the charges. Never heard any more from them.
The whole thing was a bit odd, but I can't help but think letting them know early saved us all a whole bunch of hassle, and maybe they'll fix their registration flow.
Imagine parent telling some underground rebel group that their revolution would be more successful if they organized it with Jira.
Meanwhile, this concern is so far away from the rebels, who are doing just fine with pen and paper, and are more concerned with basic needs like surviving undetected.
People are of course excited by this initiative, and wish to contribute how they know. Except what they have is hammers, and there are no nails to be seen.
It looks like you are helping, but you are only diluting the focus from what's important to what's easy to mindlessly talk about in a forum.
It looks like people are trying to organize how to organize, instead of actually organizing anything. It's like the difference between being a writer and a word processing expert.
You can't optimize what's not there.
It sounds like the point that you are making is reasonable, though, and unfortunately one that I see play out with a lot of FOSS projects as well. I remember a talk one time where a project lead essentially made the point that every new talk is met with a lot of "I'll setup CI for you" and "I'll setup JIRA for you", but that none of the people who say those things end up contributing code or issues.
For some reason there is a natural desire among some to organize the organizing before the thing to be organized really exists.
Contributions are all well-intentioned, but they cost resources, especially if you're not great at ruthlessly filtering out, or don't want to, for any reason; they generate a lot of heat where this energy can't be used.
Also well-intentioned contributors will set up grandiose structures, with no intention other than "to help", but no actual will to carry the actual work out. This usually turns out a wasteland after a while, which is not so much a problem until you realize you have to support it; or worst, it over-shadows the original, leaner-but-actually-productive intent.
> For some reason there is a natural desire among some to organize the organizing before the thing to be organized really exists.
I think this is why we have so many engines which have no games written for it :D
It is very easy for software contributions to create lots of friction and your analogy to heat and energy loss is really great, IMO.
Most people likely know countless ways they could help, but the time to do so doesn't match up with the need that's out there.
More to the point: where can someone who has a totally unique skillset help solve a specific problem?
The assumption I'm making (which may be faulty) is that everyone has a unique and valuable skillset that may only apply to specific problems. (Like for example, I know nothing about B list celebrities from 1950s Hollywood. But if someone had a problem to solve that involved knowledge of that period, they could post the problem and match it to a profile of a Movie professor, or just a regular person who happens to know alot about that subject. The technical complexities of proving someone knows what they are talking about and their authority can be trusted, basically filtering spam contributions, is the biggest technical challenge. But it seems like new tools like machine learning and neural nets could help us here.)
We all know we can help do basic low-skilled stuff: donate money, volunteer at homeless shelter, build houses habitat for humanity, volunteer at local garden, trash cleanup etc.
What website do I go to that collects all of these ways to help and more?
For ex, what website do I go to for helping in these scenarios:
1) I understand politics and want to work on bills in various countries to stop climate change?
2) I understand biology and want to cure the algae bloom I saw in my local lake yesterday?
3) I understand the physics of mechanical design and want to help fix a design flaw in a pair of garden shears that keeps cutting my skin between my thumb and first finger?
Or approach it the other way around: what site can I go to, register my interests and skills and get assigned to existing projects that will change the world? Where I can help anytime I have free time. Or get an assignment in my email?
Scenarios:
1) I sign up on the site saying I have a biology degree and live in Napa Valley, CA and can do environmental stuff. I get assigned to take Cesium 137 readings to followup on the Fukushima disaster in wine country at various GPS coordinates and get sent a geiger counter, instructions, and training.
2) I have Crohns disease but am willing to wear a data collection monitor on my body and send the data to multiple companies developing new medicines or treatments.
3) I care about elephants and can volunteer to take a shift watching through the eyes of an automated drone that scans for poachers on the other side of the planet?
There are plenty of ways to do basic help in a semi-organized manner to treat the symptoms of chronic human problems. There is no site that I've seen where solving the largest problems of humanity can be crowdsourced.
Where I can I volunteer my services to help design water desalinization plants to make them 100X cheaper?
You can see some historical examples, both recent (Mirai had some viruses that went around closing the bug), as well as further in the past (there's one that escapes me, it must have been around 2010?)
I wish I could cite more, I'm going to spend some time researching this and make a list for myself, it's surprisingly interesting!
[0] https://www.bleepingcomputer.com/news/security/brickerbot-au...
The winbox protocol supposedly runs over TLS and requires a username/password before anything is possible so I thought it should be safe enough, but through this bug anyone can download any file with no authentication (and the user db was storing passwords in plaintext which certainly didn't help)!
The web server vulnerability, sshd vulnerability, the smbd vulnerability - all are their fault. Had they used standard, well-tested open source packages there would be no problems, but they had to write their own custom implementations of these protocols for "reasons". I hate to think how many remotely exploitable bugs are lurking in their ipsec implementation.
Leaving a management port on a router open to the entire internet is a very bad practice. Would you leave an RDP port open to the world?
If you require remote access, at least restrict it to known management IP addresses.
Now, of course, at least bothering with CYOA is expected in security, but is rarely implemented up to snuff.... But then again, security is a "cost center", no?
Unintended system interactions are bigger in my opinion, since they tend to combine bugs across systems, or they even combine multiple unintended system interactions into bigger and more complex unintended system interactions. These things grow wild - some of the things people do with meltdown, rowhammer are wild and just enable even crazier things. On a higher level, things like server side request forgery, dns rebound attacks to circumvent firewalls are powerful tools to make existing attacks more powerful. I'm no where near an expert, just an interested admin, but a lot of these mechanics are wild.
Now where's the point to all that rambling?
Point is, most software is written and grown in very uncontrolled ways. Software outside of aviation or the space sector is written to get done, and if bugs occur, they do occur. A lot of software systems are running huge stacks with massive components - again to get done - and no one is scrutinizing all of the interactions going on in there.
With my product hat on, that's fine. Selling things is a good way to get paid. But from a security point of view, most software systems are just waiting to grow big enough until the right people care and it'll be ugly.
This is also why I largely consider our application servers to be overly resource hungry remote shells. Puts me in the right mindset.
This applies to physical security also.
There are way too many attack vectors for you to plug every possible hole.
20 years ago do you think anyone was considering that you could determine the contents of memory otherwise inaccessible your process just by reading the memory accessible to you in certain manners (Rowhammer, [1])
Or that a device taped under your desk could read your encryption keys right out of the air? [2][3]
Or that an attacker could intentionally cause errors by overclocking/undervolting "glitching" your device to cause it to skip certain instructions in order to gain access to it? [4][5][6]
Or that exploiting flaws in the way a CPU tries to predict the next instructions could lead to privileged information leakage? [7]
A sibling commenter hit the nail on the head. You have a large surface area to protect. They only need to find one tiny crack.
But by far the most common vulnerabilities are simply someone not properly validating input[8][9][10]<-(a ton of specific attack incidents listed here), or not allocating memory properly [11]
[1] https://en.wikipedia.org/wiki/Row_hammer
[2] https://www.theregister.co.uk/2015/06/20/tempest_radioshack/
[3] https://www.tau.ac.il/~tromer/radioexp/
[4] https://toothless.co/blog/bootloader-bypass-part1/
[5] https://av.tib.eu/media/32392
[6] https://www.multichannel.com/news/black-sunday-fix-dbs-pirat...
[7] https://www.wired.com/story/foreshadow-intel-secure-enclave-...
[8] https://www.pcworld.com/article/148007/security.html
[9] https://blog.detectify.com/2016/04/06/owasp-top-10-injection...
[10] https://codecurmudgeon.com/wp/sql-injection-hall-of-shame/
[11] https://engineering.purdue.edu/ResearchGroups/SmashGuard/BoF...
Using source address lists with short timeouts it's also easy to set up port knocking - first port connection attempt adds to "Knock1" for 5 seconds, second port connection attempt from an IP on "Knock1" adds to "Knock2" for 5 seconds, (repeat for X knocks), connection attempt from an IP on "KnockX" adds to "Fully_Knocked" for (duration) (or "none static" for a permanent add). You can also do both a temporary add with a duration and a separate "Has_ever_knocked" with no timeout to build a list of all remote IPs that have ever fully knocked.
The UI could certainly be more friendly, but I think that's because they're avoiding having things that can only be set up from the command line.
A quick look at the system told me the root account had no password set. So I tried a telnet and wham - I was in.
A 'who' showed someone else was logged in. So I sent a broadcast message to them saying they need to secure their system better, and logged out.
I had the habit of using an obscure hotmail email to log into FTP servers, and had done that here. I was surprised to see an email from the owner - wanting to know how I found his system, etc. He was nice, but I didn't reply.
Noooooo. Story started off well and then you tree fiddied me!
I'm pretty sure the FBI would love to arrest him by now. Just like MalwareTech.
A better solution would be automatically checking for updates, and then sending an e-mail notification to the address associated with the router's owner/sys admin.
I "registered" my router and email address with Netgear about a year ago and I was shocked a few months ago when they actually sent me an e-mail to let me know that a new firmware update was available for my router.
And having the ability to disable them and apply updates manually, combined with some forewarning like you are talking about (an email that says your router will restart tomorrow at 3am unless you do it sooner), would go a really long way.
Those of us using them on our corporate networks might be inconvenienced by a temporary outage, but that's unlikely at 3am... however, scheduling and doing these manually is still the best way for enterprise gear.
Split the difference - email the user that an update will apply on $date unless they do it first, or if they delay it (and don't let them delay it indefinitely).
I can't speak for all WISP's, but we only had service about 12 hours a day, less if it was raining. Five minutes to reboot a router would have been invisible.
Maybe the the trigger for the automatic reboot could be more complicated than just a time-based trigger. Something like
Reboot when
localtime > 2AM &
localtime < 5AM &
traffic averaged over the last 5 min < 5kbs
Basically reboot unless the router detects the network is being used actively.Automatic updates should be the default, but you should be able to shut them off if you want to make a different tradeoff.
Yes, it's extra work for developers, but the result of not doing that is the present situation - a lot of users, including a surprisingly large population of non-tech-savvy people, will go out of their way to shut down automatic updates, to avoid having to deal with broken workflows, upselling, ads sneaking in, and forced reboots in the middle of a business presentation or a game (or a surgery).
You're not going to define one set of secure-by-default rules that's going to work for everyone. Rather, you want to try to define a set of secure-by-default rules that work for most people. Then but the burden of reconfiguration and maintenance on those with unusual needs, rather than the majority.
Automatic updating could be crippling to ISP operators (Mikrotik's are very popular with WISP's, and other smaller ISP operators).
> Basically reboot unless the router detects the network is being used actively.
For the average Mikrotik router, deployed at some WISP or small ISP, that's unlikely to happen.
And kernel updates can be made faster with kexec so you don't have to reinitialize the hardware. The flashing procedure itself could also be made interruption-free with dual flash, which most sytems have anyway to avoid bricking the system.
It would take some effort to make it fast, but I think the update interruption could be brought down to the second-range. You'd still lose NAT state but that would only affect long-lasting sessions like SSH.
Other vendors have similar issues, so it's not just a Mikrotik problem, but while with other vendors it's uncommon, I almost expect it from Mikrotik.
One of my IT guy's mom complained about her machine being slow and having "too many pop ups", so he planned to go to her place on the weekend and fix it. She called back a couple of days later and told him not to bother as it was "all fixed now".
o.O
I lent him one of our loaner laptops and he brought in her computer back and put it on our test DMZ to see what was up. Yep, somebody had scrubbed all the malware and "search bars" off the machine and installed a free anti-virus package. The exceptions on the anti-virus made it easy to track down what was happening; it was set to send spam every night between 1am and 7am but otherwise was pristine.
My colleague had to do some serious soul searching before he decided to wipe it instead of just returning it ...
A system that requires users to opt in to security is not a secure system.
https://www.newegg.com/Product/Product.aspx?Item=9SIAEFK6MH2...
https://www.newegg.com/Product/Product.aspx?Item=9SIAEFK6D86...
The lock on your house door is 'opt-in'. If you don't lock it and someone steals something, is it the lock manufacturer's fault? The home builder's fault? Did they construct an insecure house? Ignorance of the proper operation of the lock is not an excuse not to lock it and doesn't shift the responsibility to someone else.
I don't know why people happily assume everything in IT is secure by default when in the real world, almost nothing is secure by default.
The chances of someone infiltrating a properly designed(and maintained) environment without detection are very slim.
Defense in depth.
It's quite an easy step by step guide.