Intel patches new ME vulnerabilities
blog.ptsecurity.com
blog.ptsecurity.com
Leaving those backdoors open in older products should lead to a recall because the flaw was there all along.
Whether it is the unfortunate materialization of Spectre-style bugs or the deliberately insecure-by-design ME, Intel's inability to support its products is dismaying.
We have some 7 year old Dell servers that are still chugging along, performing their duties as well as ever.
The main motivation for upgrade is software support (usually for the OS, driven by Microsoft), or failure rates for the older systems.
And that's for the desktop side. For servers they tend to be taken out of commission when the service they provide is migrated to a whole new platform and the legacy one goes to a better place along with the metal. Servers are more reliable than your regular desktops, they get better support, and most companies don't upgrade running systems.
P.S. I don't know of any enterprise environment where ME is used for managing servers. It's usually ILO, ILOM, DRAC, IMM, RMM. I think ME is mostly for desktops or SOHO, Intel has RMM for servers.
Edit: a quick search yields a common core or division at Intel behind the ME and SPS, so that makes a bit of sense. There has been at least 1 exploit in the wild for that SPS, it also lists TXT and ME, so I guess it's a shared (MINIX?) kernel that had the bug.
Or when the service contract expires or is too expensive to extend. You can't run a server of any importance without a service contract; it can be the difference between all the server's users and services being down for hours or a more than a week, and between IT management keeping their job for hours or longer.
It's hard to imagine not being able to find a replacement in more than a week, especially if one skipped service contract and just bought a spare or two with a fraction of the savings.
It's why moving to cloud infrastructure can be so much cheaper for these shops.
Of course, if it is a trivial size, like a single server at a small business, that's a different story.
> makes no sense for a commodity hardware installation beyond a trivial size
It's not the commodity hardware - x86 servers have been mostly commodity hardware for decades - it's virtualization (or other rapid recovery and migration tech) that makes it work.
My assertion is that service contracts for hardware have zero value for commodity hardware installations (beyond trivial size). To whit, they are, invariably, a scam.
Server hardware fails vanishingly rarely, with specific, notable exceptions of certain components. Those exceptions have predictable [1] failure rates that are therefore straightforward to budget and/or engineer around. Managers don't know to insist on this, so the scam persists.
This was true even before the advent/popularity of virtualization (and therefore "cloud" techniques).
It's also true regardless of whether or not one keeps spares on hand. With commodity hardware, vendors always have plenty of spares.. one just hasn't purchased them in advance, so there's an increased latency (not entirely unlike the spares available under a service contract).
> It's not the commodity hardware - x86 servers have been mostly commodity hardware for decades - it's virtualization (or other rapid recovery and migration tech) that makes it work.
That's where I disagree, at least partly, if I understand correctly the kind of tech you're referring to.
Those software solutions are generally just about convenience, or, ideally, reducing recovery time in the case of a non-redundant architecture. Even then, they might turn hours into minutes, not weeks into minutes.
More importantly, it's not that software that makes this work. It's the commodity hardware (and, arguably, commodity software/firmware hiding within) that makes it work. If a disk fails, any same or larger [1] can be used as a replacement. Same for RAM or even full servers. It doesn't have to be the same brand because that's the point of it being a commodity. That fact completely eliminates any problem of being down for a week due to hardware failure. In a major tech hub city, getting something delivered same day might not even require paying a premium.
Before virtualization or any similar abstraction layer, one could just move disks to a new server (best case) or restore from a backup (worst case).
[1] Other than "black swan" events like the flooding in Thailand that wrecked predictability (and reliability overall) for a generation of HDDs. Even then, if one made assumptions based on warranty length changes, those would have been close enough.
[2] Well, OK, one does have to be careful to meet minimum performance, especially for SSDs, but even a failure there won't kill basic functionality
https://aws.amazon.com/blogs/aws/ec2-instance-history/
I didn’t have one, but it seemed like a worthwhile thing to have and so I spent a few minutes putting the following list together (these are all announcement dates):
October 2007 – m1.large, m1.xlarge.
May 2008 – c1.medium, c1.xlarge.
October 2009 – m2.2xlarge, m2.4xlarge.
February 2010 – m2.xlarge.
July 2010 – cc1.4xlarge.
September 2010 – t1.micro.
November 2010 – cg1.4xlarge.
November 2011 – cc2.8xlarge.
March 2012 – m1.medium.
July 2012 – hi1.4xlarge.
October 2012 – m3.xlarge, m3.2xlarge.
December 2012 – hs1.8xlarge.
January 2013 – cr1.8xlarge.
November 2013 – c3.large, c3.xlarge, c3.2xlarge, c3.4xlarge, c3.8xlarge.
November 2013 – g2.2xlarge.
December 2013 – i2.xlarge, i2.2xlarge, i2.4xlarge, i2.8xlarge.
January 2014 – m3.medium, m3.large.
April 2014 – r3.large, r3.xlarge, r3.2xlarge, r3.4xlarge, r3.8xlarge.
July 2014 – t2.micro, t2.small, t2.medium.
January 2015 – c4.large, c4.xlarge, c4.2xlarge, c4.4xlarge, c4.8xlarge.
March 2015 – d2.xlarge, d2.2xlarge, d2.4xlarge, d2.8xlarge.
April 2015 – g2.8xlarge.
June 2015 – t2.large.
June 2015 – m4.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m4.10xlarge.
December 2015 – t2.nano.
May 2016 – x1.32xlarge.
September 2016 – m4.16xlarge.
September 2016 – p2.xlarge, p2.8xlarge, p2.16xlarge. October 2016 – x1.16xlarge.
November 2016 – f1.2xlarge, f1.16xlarge.
November 2016 – r4.large, r4.xlarge, r4.2xlarge, r4.4xlarge, r4.8xlarge, r4.16xlarge.
November 2016 – t2.xlarge, t2.2xlarge.
November 2016 – i3.large, i3.xlarge, i3.2xlarge, i3.4xlarge, i3.8xlarge, i3.16xlarge.
November 2016 – c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.8xlarge, c5.16xlarge.
July 2017 – g3.4xlarge, g3.8xlarge, g3.16xlarge. September 2017 – x1e.32xlarge.
October 2017 – p3.2xlarge, p3.8xlarge, p3.16xlarge.
November 2017 – x1e.xlarge, x1e.2xlarge, x1e.4xlarge, x1e.8xlarge, x1e.16xlarge.
November 2017 – m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.12xlarge, m5.24xlarge.
November 2017 – h1.2xlarge, h1.4xlarge, h1.8xlarge, h1.16xlarge.
November 2017 – i3.metal.
June 2018 – m5d.large, m5d.xlarge, m5d.2xlarge, m5d.4xlarge, m5d.12xlarge, m5d.24xlarge.
July 2018 – z1d.large, z1d.xlarge, z1d.2xlarge, z1d.3xlarge, z1d.6xlarge, z1d.12xlarge, z1d.metal, r5.large, r5.xlarge, r5.2xlarge, r5.4xlarge, r5.12xlarge, r5.metal, r5.24xlarge, r5d.large, r5d.xlarge, r5d.2xlarge, r5d.4xlarge, r5d.12xlarge, r5d.24xlarge, r5d.metal.
Not only can the price:performance spread vary between generations, but this can change over time, particularly because the model availability within a generation broadens over time.
Add to this the dimension of low power versions of certain processor models (whose selection is therefore strictly a cost/longevity decision, presumably invisible to someone like a cloud end user), one can't safely generalize.
The other problem is that the CPU isn't even, necessarily, the majority of the purchase cost of a server.
Specifically as it relates to the resources required to provide the same compute power for any given pool of instance capacity.
I know for a first-hand fact that AWS will not reveal its compute capacity, but in conversations with them, when we were actively monitoring the availability and continual use of all spot prices across every region, we had the data, according to them, to infer what their capacities were.
We had the data at the time, but not the interest/need, to be able to surmise ballpark compute capacity across all instance types in the spot pools - which is to say, what AWS' spare (i.e. non-fully reserved/dedicated) capacity was, and what the demand was for it. Additionally - we could have also bee publishing AWS' hourly revenues, dammit - I wish I would have thought of that - as people would have been interested in that data...
But we still don't, so we still can't infer anything at all from the announcement schedule.
Moreover, assuming AWS is growing fast enough, new instance type announcements could be entirely decoupled from deprecations. Even with both sets of data, it may not tell us anything about overall replacement rate, without also knowing the overall growth rate.
Even AWS's current mainstream "M4" offering is using a CPU from 4 years ago.
But with x86 being basically commodity and virtualization being used everywhere this is less of an issue in most cases. It means your VMs can mostly run on whatever metal you throw under them and that x86 metal can just be propped up to keep working for many, many years.
And TBH replacing servers after 2 years because you're worried about uptime feels like a horrible overreaction and self harming at the same time. Servers that are meant to provide 99.999% uptime (so 5 nines or above, or maximum 6min downtime per year) are built to run for far longer than 2 years. And they must be supported by the manufacturer of the system and the manufacturer of every sub-component for longer than that.
So unless I'm missing something I really can't see the benefit of such upgrade cycles.
I agree. Such a tactic, in the face of modern high-availability systems design (including the notion of servers being "cattle not pets"), seems actively harmful, other than, perhaps, introducing a something akin to Netflix's "chaos monkey" into the system.
I could understand pre-emptive replacement of non-hot-swappable components, but only if they're known to degrade over time [1] and only if the server is already otherwise out of service. Even then, I'd consider 3 years the minimum.
> Servers that are meant to provide 99.999% uptime (so 5 nines or above, or maximum 6min downtime per year) are built to run for far longer than 2 years.
This kind of design sounds like it's from a different "world" (mainframes) or era (proprietary, even if x86-based, Unix hardware of the 90s), not commodity x86 servers.
[1] so, maybe RAM, which has increasing CEs with age/usage, though UEs appear to be skewed toward early RAM life and therefore the bad apples are eliminated early. non-pluggable PSUs. fans. very old internal-only HDDs.
But even going some orders of magnitude lower about, 85% of x86 servers achieve at least 99.95% availability according to statistics [1].
[1] https://www.statista.com/statistics/515441/worldwide-survey-...
> Share of servers with four or more hours of unplanned downtime due to hardware flaws worldwide in 2017/2018, by hardware platform
Systems like IBM's System Z had 0 servers with more than 4 hours unplanned downtime over this period. At the other end Cisco and Fujitsu x86 servers had ~16% of servers experiencing 4 or more hours of unplanned downtime over the period.
We can infer that the vast majority of the x86 machines will be Intel considering AMD's market share in the server space.
P.S. The article is about Intel's ME flaws and patches so probably our whole discussion branched off in the wrong direction :).
Maybe that's why I was confused? I thought we were talking about commodity, x86 servers all along, particularly the GGP about "where uptime really matters" and other comments by the same person about warranty lengths (which seemed irrelevant, other than being indicative of commodity hardware).
Of course, that commenter's very terse remarks, combining "safety", "uptime", and "insurance" (and, later, billions in damages) into an environment where commodity (or not?) x86 servers are proactively replaced started me off in a state of confusion, like there was some very major assumption about the system design that I (and everyone else here, too, it seems) was missing.
I'm not sure where safety or insurance comes into this discussion at all.
People upgrade sooner because they stand to gain more than the upgrade costs not because its not safe to operate.
Your cpu is probably fit for 10-20 years of service. Every other part of your computer will fall down around it first.
But the XP example you gave kind of undermines the point. Nobody should be using XP. Not even on air-gapped networks, totally cut off, with a special support contract from MS, etc. If anyone is still using it they clearly have nothing but disregard for any kind of rules (like all the XP ATMS still out there).
I think you're severely over-estimating the criticality of stuff that's still running XP. It's mostly used for antiquated industrial hardware that gets used 5x a year, maybe (old CNC mills and whatnot), often times with no network. If it gets crypotwalled via flash-drive then someone will reinstall it and carry on with life.
So we're talking about 5 million machines with an OS designed in the late '90s (20 years ago) and that stopped receiving any meaningful updates 4 years ago.
Most of them are actually ATMs in developing countries like India and they are definitely not air-gapped [1]. The POSReady XP with the bare modicum of support until April 2019 made companies take it as a green light to keep using XP in embedded systems.
Other systems running XP: many of the NHS systems hit by WannaCry last year, many of the systems in UK Police stations, most electronic voting machines and gas stations in the US, most digital signage in train stations, airports, hospitals, or cinemas, parking garage payment machines, so many POSes, even passport security in some airports!
Does this put the magnitude of the problem in perspective? It's a threat from so many perspectives. Your safety, your data, your money, you name it.
[1] https://www.rbi.org.in/scripts/NotificationUser.aspx?Id=1131...
I'm hesitant to switch to AMD since Intel internal graphics play nicely with Linux. However that kind of doesn't matter if my machine isn't mine.
AMD has their own kind of management system available on some machines (DASH, using "smart" NICs like Broadcom), but the PSP isn't even involved when DASH is available and in use, as far as I know.
However, on my Intel machine with AMT, there's a network port opened by the ME itself (TCP/16992). It can use the same IP as the main OS, or a different IP entirely if desired. It uses the same ethernet port as the main OS though, splitting the packets that are directed to one of the ME ports and selectively allowing the rest to continue to the main OS (there's a low-level ME firewall[1]).
On that port, there is a full remote desktop with mouse and keyboard, the ability to remotely connect small drives and/or ISO files, a remote serial console, a low level firewall configuration utility, and power/reboot controls. Even on machines where AMT is not even supposed to be available, there have been PoC demonstrated using one of the ME flaws to turn it back on[2].
[1] https://i.imgur.com/Wphopk1.jpg
(slide 68) [2] https://www.blackhat.com/docs/eu-17/materials/eu-17-Goryachy...
That said the amdgpu driver is still relatively new and major changes and improvements are still ongoing but overall it's definitely production stable and for dedicated gpus performance is generally very good.
Not trying to sound like a fanboy it's just nice to see someone else push as hard for open source mainlined graphics as Intel has for all these years.
Case in point, the ATI Radeon HD 6310 that came on EEE PC models, the video hardware acceleration no longer works as it used to be.
Sure, I can probably hack the older driver into newer Ubuntu releases or track down someone that has already done it, but that is exactly what I don't want to spend my time doing outside work.
[1] https://www.ifixit.com/Guide/Asus+Eee+PC+1008ha+Graphics+Car...
The graphics card was working perfectly fine before they decided to reboot driver support.
Now with the legacy driver I have to force enable acceleration and even then I sometimes get the feeling it isn't really working, given how the fan behaves when watching movies on the go.
If you want zero-copy video playback for optimizing battery life use mpv with --hwdec=vaapi. Or vdpau or whatever API is supported with that driver. You can also try switching -vo to vdpau/vaapi from OpenGL.
> I know, and this kind of attitude regarding drivers is what as graphics oriented person, eventually pushed me back into the Windows/OS X world.
On Windows you get legacy drivers for older architectures as well. Also the case with NVIDIA. It’s legacy hardware, after all… AMD’s new driver is completely open source, but it targets GCN, which is a completely different type of hardware.
The issue is that AMD's legacy driver is not the same code as the driver it replaced.
https://www.omgubuntu.co.uk/2016/03/ubuntu-drops-amd-catalys...
On Windows and OS X, the legacy drivers keep working.
The fact that they are open source is of course a good thing whats not is that they were at one time less than half the performance.
The new drivers from AMD gpus are both open source AND performant.
Basically the proper strategy 2003-2017 was to buy nvidia and install the binary drivers.
At present you can go with either so long as you aren't buying hardware too old to be supported by the new amd drivers. I'm still using nvidia on all my hardware but maybe I will give amd a try again next time around.
It sucks that its complicated but its not as complicated as it seems.
Eventually one gets fed up and wants the laptop just to work.
Asus used to sell their netbooks with Ubuntu pre-installed on the German Amazon store.
It already had a phase where I couldn't use WiFi for a couple of months as Ubuntu decided to replace the binary driver for an open source implementation partially working, and then fix issues as they came.
Now it is the same story regarding ATI drivers.
As long as you avoid hostile vendors like Realtek and NVIDIA everything should just work.
TL;DR Ryzen 2700X is great value for money, would recommend it any day :)
System runs cooler, quieter and faster. Gentoo absolutely flies on it as well
Intel only patches its own firmware, but it's normally up to manufacturers to update that firmware for devices. So most PC/laptops users really won't even see these patches.
And I agree with your main point. For one of the more recent Spectre-class flaws, Intel basically said "it's third-party developers' problem to fix."
A regular desktop motherboard might include the correct HW but the manufacturer won't bother including the ME FW.
And companies like Lenovo, HP, and Dell already offer the updated ME firmware.
Every Intel system shipped in the last few years has the correct CPU, chipset and some version of the ME firmware (it's involved in initial platform boot, including Boot Guard which validates the BIOS before the CPU even gets a chance to run it), however some of them have "diet" ME firmware with fewer modules present.
The supported NIC is the only one that common desktops and laptops may not have at all. There's a side-channel required between the NIC and ME for certain features like AMT (remote desktop/management), and some NICs don't/can't support it. However, I recall seeing something about Intel allowing non-Intel NICs to be used at some point.
> Which you're only going to find in OEM systems marked as such - vPro.
See slide 68:
> Our Achievements
> • Switched-on AMT on non-vPro systems
https://www.blackhat.com/docs/eu-17/materials/eu-17-Goryachy...
But I was under the impression that the ME FW on non vPro systems is there just for things like BootGuard or TXT, which (presumably) wouldn't need the full ME functionality. I did not expect that some manufacturers ship the AMT-enabled FW without marketing it as such.
I can't see the actual demo so I imagine they found some non-vPro system where the manufacturer actually shipped the full AMT FW. I'm curious if that's a common occurrence.
Non-AMT firmware is about the 3-4 time smaller than AMT-enabled firmware. That's a hefty difference so if you have the non-AMT version and you can still somehow enable it you'd get a very limited subset of what AMT usually is.
You could get the impression that every single computer with a Intel CPU is vulnerable to be hacked over the network. Which I really doubt.
> I want them to disable that thing by default!
When you say this is enabled by default, do you literally mean that they open up a HTTP server without you doing anything?
I don't know much about the Intel ME but the things people are saying about it just seem totally unbelievable.
It’s in most business notebooks (HP, Dell, Lenovo, etc.)
Things that come with great benefit to risk for abuse:
- Intel ME
- Computrace
- Device Guard
Consumers have also shown zero intention of paying more for secure devices. Until we have a public ME hack with real consequences, I do not expect that to change.
It's used to remotely manage almost all aspects of the computer; it's a parallel, out-of-band subsystem, complete with its own processor, memory and OS.
It's very useful for managing computers at scale and at physical distances. Imagine making changes to thousands of computers; manual, one-at-a-time, hands-on solutions are very inefficient and error-prone. Imagine a campus or office building where the average distance from the IT support office to the computer is 20 minutes. Staff can spend most of their time in transit: 20 minutes there, 10 minute fix, 20 minutes back. 80% of the IT labor budget is paying people to walk.
But I agree; there's no reason the computer's owners shouldn't have the power to disable it if they choose.
Unless you have a fuse to be pulled, or blown.
I wouldn't have purchased my last Intel chip with better knowledge of this junk side-loaded, full-access, completely-opaque OS, made by a company incredibly thick with our mates in the 5 Eyes etc.
It's like a branch in your code that does an escalated execution of any input.
Yes it's good to put a fence and prevent it but the better route is to excise the logic from your code base completely.
CVE-2018-3628 - "Buffer overflow in HTTP handler"
Affected processor list (simplified reordered by me to reflect relevance and improve readability):
• Core i3/i5/i7, generation 1-8 (that is, all of them)
• Xeon E3-1200 v5/v6
• Xeon Scalable
• Xeon W
• Core 2 Duo vPro, Centrino 2 vPro
Nothing about me opening a text editor, interpreting most code, or compiling the occasional thing require anywhere near that much power.
For the overwhelming majority, and i really mean overwhelming, there's really no progress or point to upgrading anything except the gpu in the past 7 years or so
in laptops, new stuff uses a LOT less power for the same speed, so there's that I guess.
Yeah, you could argue that doubling the core or thread count doubled performance in selected software but the reality is that for real world use, excluding specific corner cases, the improvement is hard to notice.
A reason to upgrade is to have a newer platform and the features that would bring, definitely not the CPU.
https://img.purch.com/r/711x457/aHR0cDovL21lZGlhLmJlc3RvZm1p...
Literally 2x.
But all-up, Windows 10 at 6 Watts. Including the display, storage, the whole system — or so the performance counters on the battery tell me.
The extent to which this particular strategic business option was discussed during the design phase of the ME is hard to know from the outside. Surely someone within Intel pointed out that the ME was inherently insecure, but as for all the consequences, who knows?
Planned obsolescence through major security bugs doesn't sound very smart to me.
I don't get a vibe that Intel is enjoying this publicity or the presumed replace of those chips. My understanding is that AMD is quite competitive today in the data center.
Wasn't the C2xxx series also an expensive server SOC. Man these guys are fumbling.
[0]https://www.theregister.co.uk/2017/02/06/cisco_intel_decline...
So if you're using an Intel CPU today you'll just have to pick your poison.
Intel® Atom™ Processor C Series
Intel® Atom™ Processor E Series
Intel® Atom™ Processor A Series
Intel® Atom™ Processor x3 Series
Intel® Atom™ Processor Z Series
This being said, this is the current status for my old Atom N270 (2008): https://imgur.com/a/pbeJ306
Unless the tool is wrong, but I think it was generally marketed as a reliable source. In which case maybe someone can recommend a reliable one to test the vulnerability.
If you want to truly test speculative vulnerabilities, compile this program: https://github.com/Eugnis/spectre-attack (EDIT: although this one probably won't work on N270, since it uses rdtscp, that it doesn't have, need to find version with just rdtsc)
The Web really has won.
We have HTTP parsing failures in our CPUs.
> It's not like the hardware implemented an HTTP server.
But yes, yes yes it is.
I don't mean "in our CPUs" in the sense of running nginx. I say "in our CPUs" because
- the Web server is physically inside the CPU die
- you can't remove or change it thanks to code signing, so (to me) it's truly wedged in there
In effect, it's as hardcoded as the electrical circuitry and the transistors are.
Andrew Tannenbaum penned an open letter of surprise and shock because the use of MINIX was an implementation detail ("the licensing works for us, and it's been stared at by tons of professors and a bunch of smart kids for about 20 years, it's good"). Okay, so it's running on 3 little 486-and-a-bit class x86 cores, and it isn't embedded into the main execution pipelines (...yet. I can see that being attractive, something something "software defined ICE").
These details don't change the bigger picture - until someone can break the ME signing infra in a way Intel can't easily fix, and we can disable (or, more ideally, take over/pwn) ME for good, it's as good as mask ROM.
It's loaded from external storage, and probably runs in external DRAM.
> - you can't remove or change it thanks to code signing, so (to me) it's truly wedged in there
They literally just did, that's what the linked article is about.
> In effect, it's as hardcoded as the electrical circuitry and the transistors are.
If your criteria for hyperbole is the inability of the user to make modifications, then every effective DRM strategy is "as hardcoded as the electrical circuitry and transistors" also.
That's silly. It's a CPU. It runs software. It speaks to devices with drivers. There's no technical meat to your argument.
> It's loaded from external storage
No, it's stored on NAND located physically inside the CPU.
> , and probably runs in external DRAM.
It can access all of main memory (and actively uses this ability as part of operations, probably for MMIO communications with UEFI and SMM), but I do think the little 486+-class cores have a bit of their own dedicated RAM on-chip. This makes sense; you don't want MINIX's operation interfering with whatever OS is running, and besides, if you did use main RAM, masking the used pages (with the MMU), so the OS couldn't simply observe/control everything, would honestly leave too much of a visible dent in the system and probably make more of a stink.
>> - you can't remove or change it thanks to code signing, so (to me) it's truly wedged in there
> They literally just did, that's what the linked article is about.
Right. Now to wait and see how long this jailbreak lasts for...
>> In effect, it's as hardcoded as the electrical circuitry and the transistors are.
> If your criteria for hyperbole is the inability of the user to make modifications, then every effective DRM strategy is "as hardcoded as the electrical circuitry and transistors" also.
I'm taking into account the specifics of this particular scenario. I'm aware of other context, but in this case I'm not generalizing. ME updates are signed, and there's currently no complete "perfect" jailbreak, so the effective summarization is "it's locked down".
Considering the specifics of other DRM implementations, well, my favorite DRM is WideVine, since that runs on Linux, there's nothing like HDCP for audio yet, so... https://news.ycombinator.com/item?id=15796420 :D - but see all the replies :(, some DRM is indeed that locked down in practice, with no straightforward recourse.
> That's silly. It's a CPU. It runs software. It speaks to devices with drivers. There's no technical meat to your argument.
Technically you're right, for a strict/narrow definition of "CPU" that describes an abstract bridge between a perfect software environment and the squishy/vague real world. I'm not using that definition. I'm also not looking at Intel products as CPUs here (or, okay, not just CPUs), but as devices that contain a component I literally cannot control, with the exception of some vulnerability PoC code that's already been patched. The established status quo is that I cannot own this part of my hardware, that I'm buying the physical package but relinquishing [control over] some aspect[s] of its operation[s] to the manufacturer['s agenda].
The ME runs Minix, which does use drivers, sure. But that's back to looking at hardware exclusively from the software side of things, which is not the basis of this argument.
You could have done it with SNMP, but one of the developers got a real simple HTTP server into a smaller runtime, and it simplified the job for writing the front end.
Intel put the Management Engine into every CPU with no choice from consumers to opt out. That alone is fairly surprising, since they knew it was a big chance it would have exploits and consumers would have no defense.
But nobody reacts. Nobody cares.
At any rate I'm unlikely to provide one. Personally I probably won't care until a Snowden-like disclosure that demonstrates exploitation of the ME in a scenario that directly affects me. i.e. I wouldn't be surprised that nation states are exploiting this in targeted fashion, but nation states already have all kinds of ways to get my data in ways that I'm unlikely to be available to defend myself against if I'm targeted.
I'd be more interested in hearing about how ME vulnerabilities are being used in US State-level dragnet surveillance or perhaps to subtly manipulate the population by changing their Google/Twitter/Facebook results or something of that nature. If you find any evidence of that, let me know.
A good exploit is kept as invisible as possible, and when it is publicized, it may already be game over. A Trojan usually takes a lot of measure to stay undetected. Much of the recent crop of router-targeted exploits did not manifest their presence to the users in any way. Even with Snowden revelations, what has been shown was not just illicit mass data collection, but also huge reams of data already illicitly collected.
Also people just don't understand what the ME actually is. It is surprising the HN community who are mostly technical don't see just how atrocious it really is.
One thing to note however, is I tell my none technical associates there is a second computer in their laptop. It runs software meant to remotely control their computer. They they can not remove it, they can not see what it does, and there's frequently security bugs found in it. When I tell them this I get told I am paranoid and being stupid. However when I ask them to just imagine if it was true, wouldn't it be awful, they mostly agree.
The issue is people really don't want to know the truth. It reminds me constantly of a quote from the matrix:
"Many of them are so injured, so hopelessly dependent on the system, that they will that they will fight to protect it."
Purism[0] sell nice MBP-style, Debian-based laptops with modern Intel processors with the NSA's 'High Assurance Platform' bit set, and as much of the ME code removed as possible. It still runs briefly at boot, but this is the most-disabled you can currently get on any i3/i5/i7 processor[1].
[0]: https://puri.sm/
[1]: https://puri.sm/posts/deep-dive-into-intel-me-disablement/
System76 are planning to do something similar[2].
[2]: http://blog.system76.com/post/168050597573/system76-me-firmw...
The last Intel processors where the ME could be removed entirely without bricking, were the non-AMT Core Duos (2008ish), which were used on the Thinkpad T400 (good for your biceps) and the X200/X200T (thick, but compact, even by today's standards).
Various companies (most prominently the Ministry of Freedom in the UK) sell these models with the ME completely removed, and a completely Free Software boot process via LibreBoot (a subset of coreboot). You can find a full list of suppliers on the FSF's 'Respects Your Freedom' hardware page[3]. Most of them will also remove the ME from a compatible laptop you send them, as a service.
[3]: https://www.fsf.org/resources/hw/endorsement/respects-your-f...
These machines are also 'naturally' resistant to both Spectre and Meltdown, and obviously have no ME to exploit. None of the Intel horror-shows of the last few years seem to have touched them.
I previously thought that running an old-machine for largely hypothetical freedoms was bizarre. After these CVEs, I'm beginning to re-examine how bizarre it really is. And I do miss those old ThinkPad keyboards :)
Purism is a bit more "extreme", trying to create an all-libre laptop that avoids proprietary software and hardware. Once their equipment is a bit more stable it's something I'm going to look into more, for now they're still in development though.
https://web.archive.org/web/20180407232908/https://puri.sm/p...
Buying Librebooted machines isn’t directly supporting Intel.
I’m interested in something like this (though maybe not as beefy) for self-hosting.
Are there any companies that sell the gear you describe, or guides/wikis on getting set up?
If you are on a tight budget however i recommend buying the motherboard on aliexpress (for about $200) and if you are not comfortable in flashing libreboot yourself you can ask a company that delivers bios chips for this motherboard to flash a custom bios and supply the libreboot rom binary to them. You can easily swap the bios chips yourself. The libreboot and coreboot websites have lists of hardware that are compatible (ram/cpu).
Also if you are interested in newer liberated hardware, look into the Talos II. They provide a proper workstation that comes with only free software. It will be more difficult to setup since it has a different cpu architecture, but it is definitely the way forward.
Sadly, it seems like its going to need a major incident before people start to pay attention. I can't believe I'm actually rooting for the black-hats to do something so terrible, it wakes us all up.
Intel will be fixing ME vulnerabilities forever. It has a huge attack surface, but too obscure to get serious resources from them.
Maybe other cloud providers, and/or private clouds, would consider that.
IBM Begins Power9 Rollout with Backing from DOE, Google https://www.hpcwire.com/2017/12/06/ibm-begins-power9-rollout...
Google's Data Centers Now Have IBM Inside https://www.fool.com/investing/2018/03/22/googles-data-cente...
Introducing Zaius, Google and Rackspace’s open server running IBM POWER9 https://cloudplatform.googleblog.com/2016/10/introducing-Zai...
They're also a platinum-level member of the OpenPOWER consortium (i.e. they have a position on the board of directors).
What is crazy is that we sent people to Google about moving stuff to their cloud offering, and when I asked about the Power9 thing nobody knew anything!
Meanwhile, ARMv8 is already available to the public!
packet.net offers access to a full dedicated dual ThunderX box for $0.5/hr (or 0.1 with spot instances).
Scaleway offers KVM virtual machines (also on ThunderX) for as little as €2.99/month (€0.006/hr).
First gen ThunderX kinda sucks at single-thread performance, but you get many cores and… well, you get to start using non-x86 machines, on public cloud, right now.
If developers start actually using ARM boxes for their projects, the providers will be more likely to expand in this direction, and maybe in a year (or less?) we'll see the much more powerful ThunderX2 boxes available, and hopefully more providers will get into this…
1. https://www.wired.com/story/intel-management-engine-vulnerab...
The CPUs are actually kinda reasonable (Raptor sells the 8-core for $595, which is only $95 more than the launch price of the Ryzen 1800X).
But the lite mainboard is $1,099.
That's 5-10 times more that a board for Ryzen. And a lot is missing on the "lite".
The Ryzen 1800X is not a comparable CPU.
Certainly state actors, and militaries, have somebody somewhere being crazy mad about this and the right pay grade to act.
I suppose built-in adapters of Intel chipsets have this feature (at least if marked as vPro).
This means that there's a quite decent chance that your not Intel-branded PCI-Expess NIC is NOT AMT-enabled. Most likely your USB-attached WiFi adapter is also inaccessible to AMT.
This, if correct, means that your home machine, or your laptop, can be protected from this or any future remotely-activated AMT vulnerabilities by disabling the built-in NICs in BIOS, and using a third-party NIC, either for wired or wireless communication.
(For a server fleet, it's different, but you likely don't want to lose AMT remote access if you have a few racks full of servers anyway.)
[1]: https://software.intel.com/en-us/articles/getting-started-wi...
Outside of OEM machines the only time I managed to build a vPro enabled system was in Haswell times when Intel had desktop motherboards with the correct chipset and NIC combination, and the BIOS to run it. Right around that time Intel exited the motherboard business and most manufacturers don't bother with shipping the firmware anyway.
https://software.intel.com/en-us/node/632310
I have multiple machines with AMT and I'm actively using it every day. Some get patches, some won't because they're gen 3 CPUs or lower. Luckily I can be reasonably confident that the local network is secure so the bug isn't exploitable. This time.
I'm sure hoping AMD does better with their support of the PSP.
From the article: vulnerability enables full-blown remote code execution in the AMT process of the Management Engine.
From Intel: https://www.intel.com/content/www/us/en/security-center/advi...
Buffer overflow in HTTP handler in Intel® Active Management Technology in Intel Converged Security Manageability Engine
- The CVEs are about AMT portion only not the base IME
- Not all affected hardware will be patched (based on age)
- AMT can be disabled (and is by default)
- IME/AMT run on a croprocessor on the motherboard - not the CPU itself
- AMT runs an HTTP server for IPMI abilities
The devices which have the feature enabled are probably business devices and the feature is used to manage them. Business devices are good value targets, I guess.
Edit : I guess those devices will probably receive the fix, since they are managed
IPMI doesn't use HTTP.
AMT/vPro is apparently not for servers, and likely operates on the system NIC. The first rule of out-of-band management interfaces should be "use a physically-separate interface", which is unfortunately frequently broken (by one vendor when the procurement specified a separate interface).
>"I got another clue when your engineers began asking me to make a number of changes to MINIX, for example, making the memory footprint smaller and adding #ifdefs around pieces of code so they could be statically disabled by setting flags in the main configuration file."
Why would Intel ask Tannenbaum to make changes for them? Doesn't intel have unlimited resources?
and you can minimize ME in Sandy and Ivy Bridge, using ME_Cleaner?
edit: according to sounds' comment* in HN (2016), The ME is purportedly placed in "recovery" mode
But very stable, I am looking to flash my X220 soon for what it's worth.
https://libreboot.org/docs/hardware/gm45_remove_me.html
after that it's impossible though.
https://support.apple.com/en-us/HT208465
See the "EFI" section.
Exploiting the ME is possible even without AMT but it definitely raises the bar in the sophistication of the attack.
The me_cleaner tool might do a good job in disabling the ME in most cases but since it's doing it by removing components from the ME FW it probably doesn't work with every OEM implementation.
I don't think they've ever used the Intel NIC hardware either, wired or wireless.
The Coreboot project and the Hardenedlinux project have worked on it, and here are some resources on their progress:
https://hardenedlinux.github.io/firmware/2016/11/17/neutrali...
https://www.coreboot.org/Intel_Management_Engine
And here is a general writeup on the Intel chips and their "features": https://libreboot.org/faq.html#intel
If Intel aren't going to patch old systems, here to hoping that they will let us disable it, but it probably won't happen.
They could specify whether I'm good with AMT turned off or they went extra stupid and AMT processes packets even when it's off. My bet is on the later because otherwise they'd say to disable AMT otherwise. My next processor will be AMD.
* AMD processors do have an equivalent management engine (PSP), but I didn't hear anything about remote exploits for it.
* Beefier ARM CPUs also have something like a management engine ("trustzone" only accessible to the manufacturer). I have no idea if it has any remote-access capabilities on any common hardware. On RPi the trustzone is absent.
* Power9 does have a management engine, but it's open and you can upload your own management code. The CPU is not an option for a laptop, and hardly even for a desktop, though.
(Edited: spelling.)
Then a supplier could configure bulk orders to enable the ME and it would be left up to the customer to choose the security-for-convenience tradeoff.
Ordinary users, even if they somehow experienced the temptation to disable remote updates, wouldn't have the expertise to act on it. And any malicious actor with physical access to the machine would have other more straightforward attack vectors (like USB vulns).
So I think you're right -- and Intel has even less justification for hard-wiring the ME on by default.
For a $200 billion company, refusing to spend some ressources to patch all generations is irresponsible.
Think of how many governments still use <2012 CPUs.
We used to be answer with tin foil hat jokes about jokes on mass surveillance. Then came PRISM and nobody is laughing now.
Maybe it's going to be the same with this. We will learn many years later it was enforced by some state or economical entity that benefit greatly to have a standard unpatchable backdoor on most laptop and servers on the planet.
Out of band management is fine, but have it completely disabled by default, have it run on co-processor optimized for security, not speed, in language optimized for security, not speed and when it's on have it filter all packets that aren't signed to unique secret key. Basic stuff.
Remotely managing a corporate desktop or laptop, e.g. fixing an OS-level problem remotely, may also be important.
OTOH I'd prefer this functionality clearly delineated, usinf strong encryption, and with an explicit reliable "off" switch (preferably physical).
The proper solution is an optional management chip on the motherboard, not the CPU. PCI bus mastering NICs with wake-on-lan and other management features have existed for decades; it wouldn't be particularly difficult add the rest of the ME features.
There is no universe where what we have now is a good solution. There is no universe where there aren't superior alternatives (like custom bios for datacenter users, like just spending a little bit more money and having two versions)
If this is a problem, configure your OS better, or get a better OS. Otherwise you'll soon get need another ME on top of your ME to "fix an ME-level problem remotely".
How many layers of machines do we really need???! most datacenters already run in VMware with dockers inside running Java VM inside executing Javascript inside.
I have yet to see or even hear someone defend the ME on the grounds that they use it daily. With a few SPARC servers as the exception, we have a 100+ Intel based servers and we don't feel the need for using Intels Management Engine.
The CPU list isn't really helping as the answer is just "yes".
What remains is what, that my motherboard and NIC plays along? How can I figure that out? Is buying a non-intel NIC guaranteed to be safe?
Right about bow running pfsense on intel hardware isn't the best feeling in the world.
I ask because I've never seen its webserver on my home network. Heck, I don't even get how it could connect to the internet on a powered off device without Ethernet.
Yes, and it can't be disabled on all newer systems. The Active Management Technology (AMT) application, part of the Intel “vPro” brand, is a Web server and application code that enables remote users to power on, power off, view information about, and otherwise manage the PC. It can be used remotely even while the PC is powered off (via Wake-on-Lan).
The ME is present on all Intel desktop, mobile (laptop), and server systems since mid 2006.
Before version 6.0 (that is, on systems from 2008/2009 and earlier), the ME can be disabled by setting a couple of values in the SPI flash memory. The ME firmware can then be removed entirely from the flash memory space. libreboot does this on the Intel 4 Series systems that it supports, such as the Libreboot X200 and Libreboot T400. ME firmware versions 6.0 and later, which are found on all systems with an Intel Core i3/i5/i7 CPU and a PCH, include “ME Ignition” firmware that performs some hardware initialization and power management. If the ME’s boot ROM does not find in the SPI flash memory an ME firmware manifest with a valid Intel signature, the whole PC will shut down after 30 minutes.
(snipped from https://libreboot.org/faq.html#intel)
So if I understand correctly, at the moment, these CPUs are vulnerable on a local network but not currently over the internet?
Fortunately then various groups have already been working to remove and nullify as much of ME as possible.
What about DNS rebinding (decade old unpached vuln)? It allows websites to issue requests onto your local subnet.
Actually... Issuing a request doesn't even require any vuln, it's not a SOP violation...
And again, this is not the main point I’m arguing. What I’m saying is that supposedly “this is something that’s enabled by default on consumer devices” is verifiably wrong.
And I was trying to address this point:
> Even then, every setup I’ve seen have AMT (a separate Ethernet interface) behind a firewall and is only accessible via local network.
In the past, manufacturers used that defense when a security researcher approached them about a problem and they justified the lack of patching by saying things like "nobody would put this on the Internet". There are simple things a manufacturer can do to encourage good security by the end-user (ex. showing a warning). I don't believe that blaming the end-user is a viable path to fixing the problem. This issue isn't specific to Intel but I would prefer it if the vendor implemented more security safeguards to prevent users from inadvertently increasing their attack surface.
This talk helped me: "Replace Your Exploit-Ridden Firmware with Linux"
Ergo, could the computer manufacturer release a firmware update providing such an interface option in firmware setup if they really wanted to? Or are they stuck once the product is released?
The biggest problem I have by far with any of this is that it's not trivial to update all firmware involved. Everything else is forgivable, people make mistakes.
This isn't like some subtle software bug that went undetected. The consequences were known and Intel deliberately chose them.
This is the "not trivial to update" part. The "mistake" part is all the vulns that come out.
Plus you should take advantage of the fact that the 450 is still supported and gets a ME FW update.
Yesterday I updated all my machines with Gen 4 CPU with new BIOS, new ME FW, and (surprisingly) new TPM FW. The Gen 3 CPU machines barely got a BIOS update for Metldown/Spectre and that's it.
And as long as you are not using it and don't need it you might as well disable and unconfigure it in BIOS.
Of course in this state your machine is ready for reconfiguring it and it will accept the default "admin" ME password. Which means you have to make sure you have a good BIOS password. This will prevent someone who has 2 minutes alone with your machine from reenabling and configuring it without you even noticing.
When first news of IME being compromised, I was surprised that Shodan showed such a small number of machines.
I'm still not sure if it was an early 2000s fad that nobody really thought about, or it was deliberate (and if so, why).
Unfortunately many of those places historically didn’t have things like separate management LANs, good filtering, etc. because everything was setup around convenience and the desktop support people probably weren’t security experts.
The reason they don't show up on Shodan is that the search engine doesn't scan private networks and you have to explicitly configure it to be internet accessible. You have to configure AMT/ME in BIOS, you have to allow the connection through your router. Very few people will actually do that.
It sounds almost too good to be true: the attack surface is removed by a BIOS switch?
Unless you explicitly expose AMT to the internet you are relatively safe as long as your local network isn't compromised.
So it can be exploited only by an attacker with physical access or at least in the same network. Shodan wouldn't show you any of these machines that have ME configured and AMT blocked from crossing your router into the internet.
That is at least a little more comforting.
How'sa about a desolder-the-chip mitigation? Or a hammer-and-sharp-punch mitigation?
* In general you cannot.
* You can try to remove ME with non-official tools like https://github.com/corna/me_cleaner
* Some vendors ship specific laptops with ME disabled (https://fossbytes.com/laptops-intel-me-chip-disabled/)
* For servers or desktops, you can plug in a separate PCI network adapter instead of using the one on the mainboard (please correct me if this is wrong or confirm it as I'm unsure about it). That would at least disconnect the ME from the network by default. But anybody could still walk up to your machine, plug a cable into the mainboard ethernet port and own you at the deepest level.
me_cleaner and/or the HAP bit, or the services offered by laptop vendors which is basically doing the very same for you, may certainly reduce the degree of attack surface and the extent to which the ME poses a threat, but it is not a complete disablement or removal, and you are still reliant on a non-modifiable binary blob to bring up your system; referring to it as removal is misleading.
Since the firmware is proprietary, it's hard to make any guarantees to what extent a reduced-size ME (via me_cleaner and/or the HAP bit) reduces attack surface in practical terms. My understanding is that even with the HAP bit and me_cleaner applied, the ME continues running at least some functionality after system boot is completed.
The next stage would be to disassemble the bringup modules and figure out what exactly they do by reverse engineering, and implement that part independently. People are working on this. So far there is no indication that any of the problematic functionality is in the bringup module.
Complete reverse engineering could at least serve as an effective audit though.