Hacker News Reader

Comment by VHRanger | Hacker News Reader

The main problem with intel is direction and process. Dan Luu pointed out in 2015 [1] that intel chips had serious bugs and, given how intel acted, it was only a manner of time before something like this popped up.

What I see happened to Intel is that once they consolidated their monopoly in the late 2000s, they lost the healthy management practices that tend to come from being in a competitive industry.

All this talk from upper management about velocity was about trying to find a way to make more money when you've mined out your current niche completely. It ended up instead opening the door for AMD to make a comeback on x86

[1] https://danluu.com/cpu-bugs/

View on HN

I think that comparing the types of bugs in Luu's post (which is excellent) and Spectre/Meltdown is a mistake. The former are mistakes caused by insufficient testing. The latter are conceptual problems that are nearly fundamental to modern processor design. No amount of simple testing would have uncovered them.

packaging meltdown/spectre together was a brilliant PR coup from intel, dispite the fact that meltdown is much more severe and really only effects Intel processors.

This way they can pretend its an industry wide problem and they are not to blame, great.

How was "packaging meltdown/spectre together was a brilliant PR coup from intel"?

It was Google who found the exploits, and Google who published the exploits in the same document & their press release.

Google is the one who packaged these exploits together. How does Intel PR take credit for this?

Google were quite explicit about the distinction between the two and the fact that the more severe of the two only affected Intel. Intel's press releases, on the other hand, used the existence of Spectre - before the full details had even left embargo - to convince people that there was no Intel-specific problem, indeed no problem at all, just a industry-wide design decision.

>Google were quite explicit about the distinction between the two

They weren't though. The technical publication by project zero [0] did distinct between the two, but Google's PR oriented article [1] didn't bother to do that. The PR oriented article packages the two exploits together saying "These vulnerabilities affect many CPUs, including those from AMD, ARM, and Intel, as well as the devices and operating systems running on them."

[0] https://googleprojectzero.blogspot.com/2018/01/reading-privi...

[1] https://security.googleblog.com/2018/01/todays-cpu-vulnerabi...

I also think it was a PR coup from the Intel side. I think of myself as pretty much tech savvy. But even I hesitated for a moment and asked myself whether AMD was affected by meltdown although I knew that it didn't. All the google searches lead to sites mentioning "Meltdown/Spectre attacks" as if they were the same thing. You can hardly find sites that explicitly state: look here is meltdown, it only affects intel CPUs and it can only really be patched on the software level and that will result in HUGE perormance losses. Basically every time your CPU accesses the Kernel Space, some kind of cache cleansing needs to be done, it has to context switch.

Instead no one seems to grasp what meltdown is, intel is feeding the media with benchmarks and says there is no real impact, users believe intel.

I read somewhere that all big companies Google, Intel etc. knew about these security vulnerabilities, especially meltdown months before. Intel made a deal with Microsoft to disclose the vulnerability on a Microsoft Update date and tell everyone that there was an issue and it was fixed. Microsoft was also supposed to make all processors including AMD to get affected by the slowdown. However Google reported 1 week earlier said it couldn't keep this a secret with good conscience. But even if that plan failed there is still a lot of misinformation caused partly by intel.

Even if you visit sites such as meltdownattack.com (first result on google with some deeper info), it says we do not know in which aspect Meltdown affects processors.

I'm on the verge of buying a new desktop, I will go with intel because I have no other cheap choice. Intel still delivers the cheapest option for me (i3-8100). But I would appreciate if Intel played fairly. All this manipulation behind the scenes is corrupting the market and intel is responsible for it.

I don't know what we got ourselves into. The RAM folks built a cartel and are keeping the prices high. Microsoft is deliberately crippling third party antivirus software. Intel is shipping us CPUs with backdoors (Intel ME), there are serious vulnerabilities that get swept under the rug, yet we have to buy intel because we are locked into the x86 platform.

At least microsoft is teaming up with Qualcomm and Apple is rolling out its own ARM processors. We need to have an alternative. IMHO Intel has been playing an unfair and an unethical game.

But here's the other thing: there is no Meltdown exploit for AMD yet. The paper (https://meltdownattack.com/meltdown.pdf, Section 6.4) is clear that the race condition at the core of the exploit exists in AMD chips, as toy examples show some information leakage. It's just that they can't get a working exploit.

Is there no working AMD exploit by design, or did AMD just get lucky? I bet they just got lucky.

It's by design in a literal sense. AMD enforces security boundaries even with speculative execution. So does ARM on all but a couple of chips.

I'm talking about Meltdown, which is about a race condition with memory access checking and out-of-order execution, not speculative execution. The paper authors do not give a categorical explanation for why they can't exploit the AMD and ARM chips (Section 6.4), and indicate it may be possible to find an exploit in the future.

It's still speculative execution: Meltdown works because you can speculatively load a location based on an earlier load which will ultimately fault. That's basically the definition of speculative execution: the second load never even occurs in the non-speculative instruction flow.

It is also due to memory access checking and out-of-order execution, since those aren't orthogonal to "speculative execution" (and in fact are tightly related).

Now we're arguing the semantics of "speculative execution." When I use the term, I mean speculatively executing code across branch instructions. I use "out-of-order execution" when instructions inside of a basic block are executed in an order different from how they appear in the instruction stream. This is the terminology I learned in computer architecture courses, it's what I've read in the literature, and it's what the Meltdown and Spectre authors use.

Well long before these attacks that term had a widely accepted definition in CPU architecture that includes much more than only execution on the other side of a predicted branch.

I can't speak to your computer architecture course and I'm not sure what literature you've read (but it's easy to get the impression that it only relates to branches since a lot of literature might only be addressing that aspect), but the Meltdown authors are clear at least (quoting from the pdf):

In practice, CPUs supporting out-of-order execution support running operations speculatively to the extent that the processor’s out-of-order logic processes instructions before the CPU is certain whether the instruction will be needed and committed.

They go on to note that for the remainder of this paper their use of the term will refer to a more restricted definition related specifically to branch speculation, since that's what they care about. That's fine, and it's good they are clear about it - but it doesn't change the recognized meaning of the term (and indeed their narrowing of the term helps confirms the general definition).

They aren't as clear in the Spectre paper, and they focus on branch-related speculative execution since that's what they care about for the purposes of their description, but they don't contradict the idea that speculative execution is limited to branch prediction.

Not that the Spectre/Meltdown authors are a particularly authoritative reference for CPU architecture terminology: these are, after all, software guys peeking into the hardware world for the purpose of putting together these attacks.

Modern "big" cores are, conceptually, executing most of their instructions speculatively, since wide out-of-order execution windows that that there is a large-degree of divergence from pure in-order and any time any earlier instruction can fault, the remaining instructions are speculative (and the CPU mostly doesn't care: the infrastructure such as the ROB are going to be used regardless of whether the current head of the instruction stream is speculatively or not).

Well we don't really know, because AMD hasn't said in enough detail, but it is entirely possible that they are totally immune to the Meltdown attack.

The crux of it comes down to how their TLB, L1 and out-of-order engine interact. Can a load Y whose address depends on an earlier load X whose entry exists in the TLB but only with the S-bit (supervisor access only) end up actually using the true value of X before it is squashed?

Clearly Intel allows it to occur, but it's not obvious that this has to be the way. The TLB, which presumably contains the S-bit, is already on the critical load-to-load dependency path (since the physical tag needs to be used to select the way) so it isn't obvious that the S-check can't happen at the same time and essentially squash the result of the load X before it ever shows up on the bypass network for consumption by Y. On the other hand, it might be slightly easier to let the X result show up up on the bypass network and do the S-check in parallel and then only flag the ROB-entry as squashed after the fact[1] - perhaps it saves a MUX in the critical path.

This is very much like unlike Spectre, where it is more or less obvious by the way that almost everyone does branch prediction and speculative execution that you can probably pull of the attacks on modern cores (perhaps with the exception of branch predictors that do a full address check to access the BTB). Here AMD has also tried to claim to some invulnerability, but IMO these claims are much weaker since it seems unlikely that the predictors cannot be trained. AMD is probably just relying on the fact the the predictors are harder to train.

[1] It's worth noting that everyone is saying that Intel only applies the security check at retirement - but we don't really know this: we only know they apply it "too late" in that the Y load can consume the result, but it could still be applied before retirement, as little as a few cycles later.

I think AMD does some checks that Intel doesn't.

> The paper (https://meltdownattack.com/meltdown.pdf, Section 6.4) is clear that the race condition at the core of the exploit exists in AMD chips, as toy examples show some information leakage. It's just that they can't get a working exploit.

Actually not. The toy example shows that speculative execution occurs past a fault. Every big CPU does that and it's not news (and it's the basis for the Spectre stuff, except with "branch" replaced by "fault"). It should probably be in the Spectre paper too or instead of the Meltdown one. This also isn't news to anybody (if CPUs couldn't execute past faulting instructions, OoO would be almost useless).

The really specific interesting thing that makes Meltdown work is that supervisor-only flagged lines can be immediately used by a subsequent dependent load. That could entirely be specific to a particular architecture decision, and so it could be that some designs are immune while still being highly speculative in the general sense.

I'm not saying that AMD had some brilliant foresight to do it this way, but it might have been a design decision for unrelated reasons or just fallen naturally out of other constraints.

As I mentioned in another reply to the OP, there is some small advantage in delaying the squash of the first load (slightly reduced complexity on the L1-load-hit path), but maybe there are some small advantages to doing the early squash too (you don't use the "wrong" value in subsequent instructions and so avoid wrongly evicting useful lines from the cache, etc).

I think we can safely assume that AMD wasn't like "Oh, we should squash disallowed loads early to avoid cache timing side-channels" - not because they thought of it (they should have), but because they would have already tested Intel chips for it and published for the huge PR boost it would give them.

One might ask why it is that Project Zero's 90 day disclosure rule was waived for Intel in this case and the were apparently given an indefinite amount of time to do nothing about it.

Why is it that if the disclosure (after it was found by someone else) was scheduled for early January that everyone was still not ready and the patches still aren't released and they are still buggy and broken?

It is clear from the fact that Google broke their own disclosure rules that they were colluding with Intel to a certain extent to manage the fallout from this.

Why do you assume Google waived their disclosure policies for Intel? I thought it was clear Google waived their disclosure policy for themselves.

Google had an enormous self-interest to avoid disclosure. At the 90 day disclosure date, their Google Cloud product was vulnerable to all three exploit variants. From their blog post, it looks like at that point they didn't have acceptable mitigations for variant 2. It took Google until December, or nearly 3 months after the projected disclosure date, to fully patch their Cloud product. Google only announced the exploits to the public after they had mitigations in the pipeline for all their products.

Reviewing the timeline, isn't it more convincing that Google tried to protect themselves first, rather than Intel?

Furthermore, if Google really cared so much about 'colluding with Intel' and having disclosures 'waived for Intel', why did they announce these exploits before Intel could release a microcode patch for their CPUs? It does not add up.

> Reviewing the timeline, isn't it more convincing that Google tried to protect themselves first, rather than Intel?

It is true that both would suffer from disclosure.

> why did they announce these exploits

Because they got outed and so they had to develop a cover story.

> It does not add up.

Many parts of the story do not add up, probably because the parties involved are lying.

> Because they got outed and so they had to develop a cover story.

Your claim is at odds with Google's explanation of why they announced the exploits prior to the coordinated disclosure date. In their press article, they state:

We are posting before an originally coordinated disclosure date of January 9, 2018 because of existing public reports and growing speculation in the press and security research community about the issue, which raises the risk of exploitation. The full Project Zero report is forthcoming (update: this has been published; see above).

Recall Google's announcement came the day after multiple reports speculating the existence of a CPU hardware bug. See https://news.ycombinator.com/item?id=16046636 and https://news.ycombinator.com/item?id=16052451 .

Google's statement on why they announced before the coordinated disclosure date is valid. There was an unprecedented amount of embargo violations prior to the coordinated disclosure date. There were multiple high-profile reports correctly speculating the nature of the vulnerabilities. Where is your evidence this is a 'cover story'? I can't see it.

> probably because the parties involved are lying.

It is trivial to accuse someone of lying. Specifically, what did Google lie about, and where did they lie about it? Most importantly, where is your evidence that Google is lying?

Oh, but it IS an industry wide problem. Keep in mind Meltdown also affects some ARM processors, not only Intel.

And Intel did not have a saying in packaging both vulnerabilities together. Researchers at Project Zero did, and with good reason, since both exploit the same function through different means.

Spectre is fundamental to processor design, but Meltdown is pretty much a bug.

It is only a bug once you have discovered that branch prediction can be used as a vector of attack. Until then, it was a perfectly valid design implementation.

It was a bug right from the original design, because it sidestepped the privilege level separation guarantees of the CPU. That the bug could not be exploited for other reasons does not make it not-a-bug.

What actual data can be extracted by a Meltdown/Spectre attack? Still need to find an answer to that, nothing online says anything specific.

Datacenters should probably be worried, but what about the hundreds of millions of users out there? Doesn't seem like a big deal, tbh - until an actual exploit is out there, why should they worry?

Meltdown allows userland native code (the Javascript your browser loads from random websites is JIT'd down to native code) to dump kernel memory.

It is worth clarifying, when people talk about "kernel memory", for x86-64 it really means all of memory, because all of physical memory is mapped into the kernel's address space. So really, meltdown allows userland code to read anything in memory.

Incorrect. When people talk about kernel memory, they are talking about pages marked as supervisor in the page tables for a particular process. That is not "anything in memory."

I think you missed the "all of physical memory is mapped into the kernel's address space"

For a typical kernel without Meltdown mitigations, the entire kernel, including that window into all of physical memory, is in the page tables of every 64 bit process at all times.

Meltdown allows applications to read any mapped pages, regardless of the protection bits on those pages. That mainly means kernel memory, which is the only page set that's normally unreadable. The kernel mapping normally includes all of physical memory.

How do people find the addresses to even start probing for stuff like this?

Theoretically, any application data stored in memory. An example might be a key used to encrypt user data server-side. The scary part of this attack is how it basically breaks the assumptions that most software was written under around memory isolation. Consider an AWS instance that shares a physical system with a different customer’s instance.

Amazon, Google, Microsoft, and basically everyone else is scrambling to fix these issues because of the potential. Basically, it’s going to take years to cover the long tail for this issue, and waiting until exploit kits are commonplace isn’t necessary to understand the potential impact.

The page from the researchers is your best source: https://meltdownattack.com/. I also recommend reading the actual papers (https://meltdownattack.com/meltdown.pdf and https://spectreattack.com/spectre.pdf). I haven't gotten to the Spectre paper yet, but the Meltdown paper is excellent, readable, and clear.

I don’t think it’s fair to call Meltdown a bug. It’s like LLVM aggressively taking advantage of undefined behavior in C. Processors, like compilers, are designed making assumptions about what they owe the user and what they don’t. The architecture manuals promise that a user space read from protected kernel memory will trigger a page fault. They don’t make any other promises.

Not calling it a bug is ridiculous, it breaks x86 memory protection totally.

What does “x86 memory protection” mean? What promises does the hardware make to the software?

That software in ring 3 cannot read memory in ring 0 unless the appropriate permission bits on the page table entries are set.

I don't think the spec actually makes that guarantee anywhere. It says that a page fault will be generated if a memory access violates page protection bits, but doesn't discuss any other potential side effects.

The closest thing I can find is Intel Architecture Reference Manual Volume 3, Section 5.1.1:

> With page-level protection (as with segment-level protection) each memory reference is checked to verify that protection checks are satisfied. All checks are made before the memory cycle is started, and any violation prevents the cycle from starting and results in a page-fault exception being generated. Because checks are performed in parallel with address translation, there is no performance penalty.

If you read section 11 on caching, the terminology "memory cycle" seems to exclude cache access. Indeed, Volume 3, Section 11.7 explicitly warns that implicit caching might happen that you would not expect:

> Implicit caching occurs when a memory element is made potentially cacheable, although the element may never have been accessed in the normal von Neumann sequence. Implicit caching occurs on the P6 and more recent processor families due to aggressive prefetching, branch prediction, and TLB miss handling.

The spec isn't really relevant to the point I'm making. I'm talking about what the purpose of the design is. Memory protection as a feature exists in order to (among other things) prevent code in ring 3 from reading data in ring 0 that ring 0 has not explicitly granted permission to read. If the implementation fails to do that, then it's failed to achieve its goal, and the whole exercise is pointless—why include the silicon at all? Documented bugs are still bugs.

Again to use the crypto analogy: An implementation of, say, RSA that uses timing-sensitive memcmp to compare signatures would follow the RSA specification. But everyone would agree that such software has a severe bug.

Meltdown is a bug in the same sense that a crypto routine that uses a timing-sensitive memcmp has a bug. Memory protection is designed to prevent ring 3 code from reading memory in ring 0 (assuming the appropriate page tables are set). The implementation fails to do that.

The fact that speculative reads don't check the permission bits is arguably a design bug, not an implementation bug, but I'd still call it a bug.

Speculatively executing code without performing memory access checks seems like a big design/architecture mistake. Not really the kind of thing that testing or verification is aimed at.

That's one of the bugs, Meltdown. The other, more insidious bug doesn't involve memory checks not being done, but rather the more basic fact that results written to cache can't easily be backed out.

Exactly. It's why you need multiple, assurance activities on projects to spot what a single one can't handle. I summarized here [1] those that produced results at various points in CompSci and INFOSEC history. Far as this, the first applications of covert-channel analysis to hardware found cache-based, timing channels in both VAX [2] and Intel CPU's [3]. That the caches leaked secrets meant any number of constructions built on top would leak secrets. So, proper response would be either one cache/core/cpu per security domain or designing a new cache that didn't leak secrets. The former was high-security's fall-back with the latter attempted by many in CompSci using partitioning or masking caches. The first was done in 2005 after Percival's work showing secret leaks. That first attempt on partitioned caches also cited prior work in real-time sector thinking similarly to boost determinism: proving determinism and covert channel prevention were always closely related. There were many designs after that for both partitioning and masking before the Intel hit.

So, the root causes of shared, on-chip resources were identified by mid-1990's, demonstrated again by later work like Percivals, being mitigated from that year onward, and ignored by CPU vendors. When I asked in the past, hardware people told me they didn't care about cache security because their sales were strictly tied to customers' benchmarks of performance per dollar and watt. Customers didn't care. Suppliers didn't care. That simple.

There were in fact (tiny) segments where customers were buying processors with more robustness or predictability. Those that come to mind were some PowerPC designs from Freescale that aerospace liked, Leon3-FT SPARC that was GPL, some smartcard components, and especially Rockwell-Collins' AAMP7G [5]. Designed with EAL7 methods from 1992, it had mathematical proof of separation at level, triplicated registers for fault-tolerance, ECC memory, and MILSPEC heat tolerance. It's used in guards to separate Top Secret/SCI info from other stuff.

So, these are old attacks with mitigations of various costs that were ignored for profit maximization by the big companies and performance maximization by most consumers/businesses who didn't buy security in general. Both old and new techniques were effective at assessing leaks and mitigations, though. They could've been used at any time, were by CompSci, were by security-critical suppliers (esp Rockwell), and even more techniques exist now for analysis [6]. Intel et al will just patch up until next attack since they and the market haven't changed. ;)

[1] https://pastebin.com/uyNfvqcp

[2] https://www.google.ch/patents/US5574912

(Note: Using the patent filing since the 1992 work is paywalled. It's the same person filing what they discovered on VAX VMM project.)

[3] https://pdfs.semanticscholar.org/2209/42809262c17b6631c0f653...

[4] https://eprint.iacr.org/2005/280.pdf

[5] http://www.ccs.neu.edu/home/pete/acl206/slides/hardin.pdf

[6] https://pastebin.com/ajqxDJ3J

This isn't just Intel. Every chip company is like this. Pre/Post silicon verification is given a narrow window in which to catch everything and if it's a non-trivial bug that requires a layer change it's probably shelved till the next revision.

I think this is less of a thing that is specifically endemic to Intel as much as it's just a "big, old company" thing. Companies become massive, they eventually lose their visionary leadership, and then they spend the proceeding few decades lumbering along just based on inertia.

These companies continue to employ a lot of smart people and have exciting things going on in some components, but the corporate culture makes it hard to get a consistent positive execution.

Intel has been sitting pretty for the last 20 years because chip design is not something that lends itself to modern patterns of disruption. I'm sure it will happen some day, but for now, Intel has kept its position mostly due to the difficulty of the niche than any intrinsic competitive edge.

The point being that Intel itself is not uniquely terrible; they're routinely terrible. They're just positioned in a space that's much more hostile to less-terrible entrants, so capable people do something that is less hard.

Reply on news.ycombinator.com