Behind the Masq: Yet More DNS and DHCP Vulnerabilities
security.googleblog.com
security.googleblog.com
How many millions did Google spend optimising the code that makes that page useless for anyone who wants to zoom in on mobile? Man, that, wow, I can't express how terrible that is, Google, really!?
Edit: they have a "web version" (https://security.googleblog.com/2017/10/behind-masq-yet-more...) without that massive UX bug, where instead you can't scroll to see the table. smh
It's well known that it's generally (and practically) intractable to find all vulnerabilities in a typical networked C program using reasonable auditing effort. If auditing worked, we would know how to secure C programs and the world would be a much different place.
Also interesting to note that 3 of the 7 in the list definitely appear to be in handling of IPv6-related parts of the protocol.
> In addition to these patches we have also submitted another patch which will run Dnsmasq under seccomp-bpf to allow for additional sandboxing.
It takes a while, but the more known projects start getting seccomp patches and this will lower the impact of exploits like this in the future.
You can get good results with much smaller implementations too. See my patch to memcached for example: https://github.com/memcached/memcached/pull/275/files
The other RCE is in the DNS subsystem, and can apparently be exploited by making a PTR record lookup to an attacker-controlled DNS server:
https://github.com/google/security-research-pocs/blob/master...
My first question is: can you induce people to make PTR lookups? I don't think there's a way to make a browser do a PTR lookup, which may limit the remote exploitability of this vulnerability.
My second question is: can the vulnerability be exploited with an A/AAAA record lookup instead? If so, that would open a very devastating remote attack vector, as a website would need only include a hidden image tag pointing to an attacker-controlled domain in order to get RCE on someone's router.
That's very good news.
Edit: never mind, he says a CNAME answer to a A/AAAA query can trigger vulnerability. That's very bad news.
2.76 was only released 16 months ago though. There are probably tons of router firmwares out there that are way older than that.
(And most of the devices out there will never get updated even after this, just replaced one day)
dnsmasq is used elsewhere, of course, but those deployments are far more likely to receive security updates.
Relevant posts:
- https://groups.google.com/forum/#!topic/kubernetes-dev/QWIzh...
- https://coreos.com/blog/dns-vulnerability-patched-in-kuberne...
Then it lets you escape to the dnsmasq container. Which does have a shell, but I'm struggling to think how this gets you more capability than "able to run arbitrary code inside the cluster".
Most likely I missed something. Help me spot it.
You would need to middle man traffic between dnsmasq and upstream DNS or have some control over DNS records and be able to force dnsmasq to do lookups. As you point out the escalation is also far more limited. Ok, so you have a shell in the dnsmasq container. Now what?
> All GKE clusters have already been patched, so no action is necessary for GKE users. Users will see a "-gke.1" version for the MASTER_VERSION of their GKE clusters, which contains the patch.
Just verified that my home OpenWrt WDR4300 is affected.
Specifically, here is where all updates are coming from: https://downloads.openwrt.org/chaos_calmer/15.05.1/ar71xx/ge...
No updates since March 2016.
EDIT: Has development has moved to the LEDE project? I don't understand what's happening in the OpenWRT/LEDE split. There appears to be a fairly clean upgrade path, though: https://forum.lede-project.org/t/upgrade-from-openwrt-to-led...
tl;dr updated packages of dnsmasq are currently building and should be available soon
Although LEDE and OpenWRT expressed interest to re-merge.
For example, the Kong variant updates for http and checksum despite repeated requests to deliver over https. I’ve got quite a few other examples but sadly on mobile right now.
I'll try to launch one of the POC later in the day, but for now I assume worst case.
CVE-2017-14491.pcap: https://www.cloudshark.org/captures/016080cbeab5
CVE-2017-14492.pcap: https://www.cloudshark.org/captures/fd4bfa9403fc
CVE-2017-14493.pcap: https://www.cloudshark.org/captures/1c30b56dc1d3
CVE-2017-14494.pcap: https://www.cloudshark.org/captures/657de976c0d1
Used this repo to generate these: https://github.com/google/security-research-pocs/tree/master...
Let's not get into the discussion of whether or not everything that can be connected to the Internet should be. (I'm in the "should not be connected camp", but I think my question can stand on its own without this discussion).
The second answer is also obvious: Try to avoid as many bugs as possible before shipping the product. If you include a highly exposed software like dnsmasq make sure you look for bugs in it before shipping it.
I don't remember having seen any professional security audit for dnsmasq before. Such an audit is probably an almost irrelevant expense for all the router manufacturers out there, yet none of them chose to fund one...
Mozilla ("Secure Open Source" [0]) funded an audit of dnsmasq [1] by Cure53 [2] about a year and a half ago. The final report [3] (PDF) lists one "medium" and five "low" issues.
It just goes to show that a "successful" audit (i.e., one that doesn't find any major/critical issues) ultimately doesn't prove much of anything at all WRT the overall security of a piece of software.
[0]: https://wiki.mozilla.org/MOSS/Secure_Open_Source
[1]: https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#d...
[2]: https://cure53.de/
[3]: https://wiki.mozilla.org/images/f/f7/Dnsmasq-report.pdf
Open source is extremely heavily used by commercial software and hardware manufacturers and the reality is that very few of them contribute back, meaningfully, to Open source security.
The end result will continue to be bugs like this, with ever growing impact.
"Tragedy of the Commons."
If all manufacturers in the space assume someone else will pay for the audit (or assume it won't get audited), then it doesn't get audited. If one does pay for it, they all get to benefit due to the nature of open source patches.
It's funny yet sad that it took Google Security acting on a bug report by a (presumably unrelated) engineer to find some of these security issues. Google has strong incentives to protect its own network (its livelihood depends on intellectual property and safeguarding data). The companies that sell mostly disposable SOHO routers have such little interest in security, yet it's literally the gateway most of us use to the internet.
At least with Android, Google is forcing manufacturers to have a minimum security patch lifetime. There's no one in a position to require or enforce that among the tens (hundreds?) of thousands of home/SOHO routers models out there.
I expect we'd maybe be able to get some of the big players (Netgear, Linksys, D-Link, etc.) to agree individually to support products with security updates for X amount of time, but it's still dicey. I used to work for one of them[0], and often the people internally who own the products don't even know enough about the software running on them to even be aware of security issues. They're entirely at the mercy of the outsourced software firm for that. The margins are low enough (and price competition high enough) that you're not going to see much effort without consumer support. That is, consumers being actively willing in large numbers to pay more for a more secure product (so, that's not happening).
If we could go by a model where everything was based on a third party, like OpenWRT or LEDE, and if the appliances could auto-update in-place from the upstream package repositories (nothing run by manufacturers or the software integrators), then that might work, but the manufacturers would have a hard time supporting it, as I'd expect there to be a lot of random breakage.
I'm not sure there's a great solution here. Savvy consumers will pick a brand that has a strong track record with security (perhaps buying SMB or enterprise-grade products), or will flash 3rd-party firmware that they can maintain and upgrade themselves. The masses are pretty much lost and dependent on a commodity industry that has little incentive to pay close attention to security, and a lot of incentive (in the form of cost and pricing pressure) to ignore all but the worst security problems.
[0] This was 2004-2009, so things may have changed, but I'm not expecting much.
Yep, I'm enjoying my OnHub.
ISPs bundle routers with their contracts. So if their customers get attacked through hardware that they supplied they will have at least some incentive to put pressure on the upstream manufacturers.
You can get the same protection NAT offers for IPv6 with an incredibly basic set of firewall rules, only allowing devices on the LAN side of the connection to establish connections and punching holes where necessary.
The downside is that they yet again break P2P protocols, although that's somewhat easier to fix.
https://security-tracker.debian.org/tracker/CVE-2017-14491 https://security-tracker.debian.org/tracker/CVE-2017-14492 https://security-tracker.debian.org/tracker/CVE-2017-14493 https://security-tracker.debian.org/tracker/CVE-2017-14494 https://security-tracker.debian.org/tracker/CVE-2017-14495 https://security-tracker.debian.org/tracker/CVE-2017-14496
Sure, we were vulnerable for a brief window, and without a doubt there were other vulnerabilities, but we were pretty much as safe as possible, let alone for a half-person team.
You just don't leave windows broken. You don't leave the holes unfilled. If you find a crack in the foundation, you fix it. To do otherwise is not only irresponsible to your peers and employer, but to your customers.
My only criticism is that often the roles responsible for such things are unpleasant, come with little respect, visibility and/or pay. Companies should just suck it up and overpay a handful of engineers and security specialists, charged with the responsibility of ensuring the company avoids being vulnerable. Think of it as insurance -- it's worth far more than a bad press cycle or having your customers exploited and rightful loss of trust. The bang for your buck is off the charts.
The "black swan" events aren't costless to mitigate. In fact, they are extremely expensive. And just like there can be a black swan for cyberattacks, there can be a black swan like a terrorist attack (like companies in World Trade Center Towers 1 and 2), a massive natural disaster (think businesses in Puerto Rico right now), massive electrical outages (say an EMP or simply just Enron malfeasance), etc.
Spending to protect against and mitigate all possible black swans approaches infinity and doesn't every pay off until the extremely rare but extremely impactful event ever happens. And in cyber security, you never know because your defenses prevented it before it ever started.
While my gut agrees with your proposal, my brain tells me that simply patching doesn't fix either the most common issues (basic fraud, phishing, social engineering, malicious insider, etc) or the most dangerous issues (infected supply chain, zero days, etc).
Companies will still pay for insurance because your proposal doesn't insure against those other scenarios, so they will see insurance as a crutch which mitigates some of the costs which come with unpatched vulns and the race to patch.
But compared to the costs associated with insurance or damage control, having a small team of security minded people that can make changes is basically free.
One line installers are the difference between a 3-day mitigation and a 3-hour mitigation. Sometimes the difference is in weeks.
It's definitely not about who "takes that level of responsibility" to patch manually, it's about what's operationally feasible. Dnsmasq is probably the widest deployed networking service for consumer devices. Now every consumer device running an old version, which includes Linux desktops, Android devices, home internet routers, etc are vulnerable. They even state how big the problem is in the release:
This software is commonly installed in systems as varied as desktop
Linux distributions (like Ubuntu), home routers, and IoT devices.
Dnsmasq is widely used both on the open internet and internally
in private networks.
Meanwhile, as soon as you publish a vulnerability, responsible network owners can make integrity vs. availability tradeoffs, even without manually managing a patch. Somehow, as an industry, we've come to the conclusion that nobody can shut a service off, even in response to a lethal security vulnerability. Better that we risk our customers data than violating an SLA. That's messed up.
But this isn't about shutting off a service to protect a company's bottom line. This is a vuln affecting consumer devices. SLA has nothing to do with it. People's home networks and devices, not to mention a myriad number of embedded devices and sensors all over the world, are going to be affected by this - not corporations protecting customer data. Affected consumers have no means to patch themselves. Releasing this before vendors could generate patched packages or firmware is just monumentally irresponsible.
This would be preposterous reasoning if it weren't overwhelmingly normalized in our industry. It's an embarrassment for the whole field.
What you seem to be suggesting is that we should simply all disconnect from the internet until vendor patches can be prepared, the reason being it's better to stop any potential and unknown exploits going on now, than stop the hundred-fold new exploits that absolutely will follow its public announcement.
You mean if hundreds of the popular vendors were notified for a coordinated release? And you think that's not making the exploit effectively public? Even skipping possible monitoring of emails, I don't expect that the set of blackhats and employed security engineers have no common elements.
Why do we put locks on our doors? It's not so that career hardened cat burglars can't get in - it's to prevent crimes of opportunity and small time crooks from getting an easy target.
I feel like there's an analog with software. Quietly commit fixes for the RCEs to DNSMasq. Those who care will stay on the latest version. You can even say "Fixes security issues" in the release notes. But, wait awhile to release the PoCs and detail the exploits completely.
The nation states will find a way to exploit it no matter what, of course. But you average cybercriminal will have a much harder time without PoCs clearly showing the attack vector.
Especially for a LAN-only type issue like this - unlikely that someone trusted with the embargo would go blackhat LAN-side here during the wait, but now every script kiddie in the world has the knowledge to make a mess while we're still awaiting vendor patches?
I don't want to single you or anyone else out about this because it's a universal problem among startups. But that doesn't make it close to OK.
I generally don't have an issue with the security decisions of people who don't know better. I do have a problem with people who should know better and essentially refuse to because of cost and convenience and availability concerns. Security should trump availability most of the time, but almost never does in our industry. That is messed up.
This, the Broadcom wifi RCE and the Bluetooth RCE are like a concurrent land, sea and air pwning of people's personal and home devices.
The reason these devices are cheap/affordable is because of the open source community. The reason the internet has made it to this current state is because of the open source community. If you want the internet to be better and safer you have to roll up your shirt sleeves and contribute in any way you can.
I suspect the consumer protection laws take similar views in many countries.
Like I said before, there are a number of available DHCP and DNS implementations in memory-safe languages. Many of them are open source. So the vendors don't even have that excuse.
DHCP: isc-dhcp or dnsmasq
DNS recursor: dnsmasq, unbound, bind, djb's dnscache, maradns
None of these are written in memory safe languages, but they're the only ones I would trust because they have YEARS of testing and getting the RFCs right.
I'm glad Arch Linux has such strict firewalls by default!
With djbdns I have never felt the need for it. Nor do I use DHCP. Or even ARP. At least, on computers that allow control over such things.
I guess I am missing out on the convenience, or perhaps there is an alternative selling point I have overlooked?
I prefer wired networks and to attach particular RFC1918 IP's to particular devices. Works for me.