Hacker News Reader

Comment by zackmorris | Hacker News Reader

Ok I think I understand the subtleties of these attacks now. But: can anyone tell me why the accessibility check for protected memory doesn't happen before the cache loads the contents of RAM? If that happened then none of these attacks would be possible.

I got my computer engineering degree in 1999 and ended up going the computer science route making CRUD apps all day. I feel in my gut that some engineer, somewhere, MUST have asked this question at one of the big chip manufacturers.

Am I missing something fundamental? Is the access check too expensive? If it isn't, then can the microcode be updated to do this, or is caching/accessibility checking happening at a level above microcode? If that's the case then it would seem that pretty much all processors everywhere that do speculation without protected memory access checks are now obsolete.

View on HN

> can anyone tell me why the accessibility check for protected memory doesn't happen before the cache loads the contents of RAM?

From what I understand, it does happen on AMD, which is why AMD CPUs are not vulnerable to the more dangerous Meltdown attack (any code reading kernel / hypervisor host memory).

Intel / ARM delays the checks until later, to the time when the speculated instructions are actually finalised and make their results available. This is faster, and loading some memory into the cache is normally invisible to the unprivileged code. The checks would still be done when actually reading that memory. But nobody spent enough time considering the timing side-effects of the cache.

Now even if the protected memory reads are fixed by the OS updates - then it still leaves the Spectre attack - code running in a process reading all of "its own" memory, regardless of any software sandboxing. This means that all sorts of sandboxing methods for javascript interpreters, bytecode interpreters, plugin architectures, etc are insecure. And the OS patches can't help here, because the sandbox isn't in protected memory.

Thank you, that was a succinct explanation of the difference between AMD/Intel and the order of speculation and protected memory access check for the Meltdown attack, and makes it easier to understand:

https://en.wikipedia.org/wiki/Meltdown_(security_vulnerabili...

I see now why the Spectre attack is so serious (reading out of bounds within the same process memory). I feel like there may be ways to catch unallocated memory access similarly to how protected memory works. But, that wouldn't help reads from allocated memory in runtime environments like for Javascript (where separate scripts in the same process space aren't meant to see each other's data). This is clearer to me now:

https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...

Going forward, we may have to assume that security is only possible with true process isolation. For example this might put pressure on OSs to fix their slow context switching implementations to encourage the use of processes instead of threads. Beyond that, I can't see any easy way to fix the situation and am highly skeptical of things like compiler fixes, because there will likely always be another way to abuse various instructions to read outside memory boundaries.

The slow context switching between processes has nothing to do with OS implementation. True context switching involves a page table flush. This is slow due to caching, independent of the OS. The only thing the OS can do is tell the processor which parts not to mark dirty, but -- as these attacks show -- this can expose vulnerabilities.

Too bad segmentation was dropped for 64-bit code on x86, leaving just page tables. 32-bit x86 retains the segmentation model of the 286, extending it to 32 bits, and making it work with virtual addresses instead of physical addresses if the paging system is also enabled.

Most 32-bit operating systems ignored the segmentation system, basically just running everything in what the old timers would call "small model".

If we had segmentation in 64-bit mode, then I wonder if we could defend against a lot of these problems by running things we want to sandbox, such as JavaScript, in a different segment that only has access to the upper end of our virtual address space?

As long as the sandboxed code cannot change the segment registers, this would prevent it from generating an address outside the sandboxed portion of the processes' virtual address space.

I don't recall if the x86 segment system provides a way to trap attempts to change a segment register. If I recall correctly, it does support more than just the two level kernel/user protection system, and I think it supports not allowing loading a segment register with a selector that refers to a segment belonging to a higher level, so maybe if user mode was split into two levels, so sandboxed code could be run at a less privileged level than the main process it could work.

In general, I think processor designers need to take into account the need for processes to run sandboxed code, and provide some kind of mechanism the processes can use to protect themselves from malicious code in the sandbox.

https://www.reddit.com/r/programming/comments/bkos1/what_has...

https://pdos.csail.mit.edu/~baford/vm/

> This means that all sorts of sandboxing methods for javascript interpreters, bytecode interpreters, plugin architectures, etc are insecure.

I just now understood the impact of Spectre. It is not just that all existing attempts to execute code in a sandbox are vulnerable. For CPUs with this problem, it is literally impossible to create a secure sandbox.

We certainly live in interesting times...

I'm still wrapping my head around Spectre. To me it does seem possible to create a sandbox secure against Spectre, but you have to recompile every program with retpoline to mitigate variant 2 and run in a separate process to mitigate variant 1. This way even if something breaks in a sandbox it will be able to see something else only from similar broken sandboxes, but not from other programs. Although there is a possibility of other side channel attacks.

(hoping for someone to correct me if I'm wrong)

I hope I understand your question correctly. The request asks for something in protected memory and also asks for something based off a portion of the protected memory (like the first byte). The system denies access to the request then puts both results in cache. The attacker then asks for a byte of memory similar to the second request, which the system tries to get from cache but then goes to memory since it wasn't in cache. The attacker doesn't want that result so cancels the request and asks for another byte similar to the second request. That process repeats until the system says "Hey this byte is in my cache" and gives the result back to the attacker. That let's the attacker know what the first byte of that protected memory was. The attacker then repeats that whole process until they've read the entire protected memory, which is at a rate of 1500/bytes a second. It never gets the actual protected memory from the cache.

this is for meltdown right? how would you explain the difference between this and spectre, going from this explanation?

> Ok I think I understand the subtleties of these attacks now. But: can anyone tell me why the accessibility check for protected memory doesn't happen before the cache loads the contents of RAM? If that happened then none of these attacks would be possible.

If that happened, Meltdown wouldn't be possible on the processors on which it is possible.

Those checks in no way mitigate against Spectre. Spectre is both simpler and in many ways more profoundly devastating -- as long as an attacker can influence the statistics that the CPU uses for branch prediction, the very act of trying to "work ahead" in another process can be influenced by the attacker, and will produce some side-effects even if you throw out the result, and those side-effects suffice to infer information about the work you did but threw out.

I think you're missing that the misdirection happens in the kernel. On an indirect branch (i.e. "jmp <register>") the CPU picks a value for <register> from a small cache when speculatively executing, since the real value isn't known. A malicious program can fill the small cache with addresses of kernel code that do some computation with kernel data and then depending on the result, touch different parts of user application memory.

There is no memory protection violation because it happens in the kernel; obviously, the kernel can read its own memory.

> I feel in my gut that some engineer, somewhere, MUST have asked this question at one of the big chip manufacturers.

Sure, they were about to commit career suicide but then they learned to love the bomb and went on with their day. Maybe they even tried to explain the problem to management but somehow it got lost in translation.

The memory access could potentially take a very long time, so it makes sense to start it as early as possible, in parallel with all the other operations. Saving a few cycles on each memory access adds up significantly overall.

Right, but at a cost of a leaky cache abstraction. Which is now hugely expensive to work around, making me question how that ever got approved if the security model was clearly leaking at that point (which has been known for the entirety of speculative execution, right?)

how that ever got approved if the security model was clearly leaking at that point

Keep in mind that for a very long time, PCs and x86 CPUs were used in environments where either there was a single user with full access, or multiple users that aren't completely adversarial (look at Win95/98's multiuser security model, for example.) Memory protection and other security features served as a barrier to accidents and to "keep the honest honest", not determined adversaries.

They still are today, but this is very different from the shared servers/cloud computing environments which have now become common --- completely mutually untrusting users with possibly adversarial relationships are sharing the same hardware.

This is the reason why all the CPU manufacturers have had some variant of "as designed" in their public comments --- speculative execution was designed with the former model in mind, not the latter.

> how that ever got approved if the security model was clearly leaking

When you consider a theoretical model of the CPU, then it's not leaking - the speculative execution, cache, other parts of the CPU are designed carefully so that no data can "escape" and be read by processes that don't have the permission to do it. Speculated execution can happen, but before any results from that are released, the permissions are checked, and if they fail, the results are discarded.

What people did not consider is the timing attacks that do leak information. It's only "clear" to us now after the attack has been demonstrated, even if it has been present in CPU design for the last 20 years.

There are probably many more of these side channel data extraction paths possible. For example, in the recent years attacks on cryptographic algorithms have looked at similar timing measurements, and in some cases power consumption measurements.

Isn't the knowledge of potential for timing attacks using cache/memory fairly old? I am pretty sure I heard of the concept long ago.

Very old in the arena of cryptography.

No one in the CPU Architecture Design arena put 2 & 2 together [1] to realize that the same side channel that was devastating for cryptography work would also be quite devastating for bypassing memory permission protections in the CPU's they were designing.

[1] probably because the intersection of "cryptographers who can mount timing side channel attacks" and "CPU Architecture designers" is very close to zero.

I may be wrong, but the timing attack only reveals the cache line, but being able to read the cache line is something itself I would expect to fail even without the timing attack. How does that not cause an exception?

My understanding is that you never directly read memory to which you shouldn't have access. You arrange for the processor to read a specific address you control based on the value of the illegal read. By timing subsequent accesses to that user address, you can infer whether or not the processor brought it into the cache, and thus infer the value of the illegal read.

> if the security model was clearly leaking at that point

The leak is only clear in retrospect. Many, many things are only clear after you see how they were done.

It has been twenty years since processors with this vulnerability started appearing. Over those two decades, thousands of very smart engineers (including state-sponsored ones) have collectively spent millions of hours of analysis trying to find security flaws.

No one has found such a clever timing attack until now. So "clearly leaking" wasn't "clearly leaking" until this week.

> The leak is only clear in retrospect. Many, many things are only clear after you see how they were done.

However, even at the time of the design it would have been obvious that deferring security checks is a risky design choice.

No one has reported such clever timing attack until now. There's no reason to suppose that such an attack hasn't been used by nationstate actors for, say, a year already.

Colin Percival found a similar problem with Intel's implementation of "HyperThreading" back in 2005:

http://www.daemonology.net/hyperthreading-considered-harmful...

In retrospect, yes this is of course a big problem. But regarding how it could ever be approved...

Until this news broke, a CPU designer would tell you that speculative execution is a well-understood, proven approach to gaining a lot of performance. Branch predictors are really good at figuring out where the next instruction is going to come from, and are a really important tool for avoiding stalling out the whole machine while you wait for the next instruction to come back from memory.

And intuitively, it seems really "safe." All you're doing is having the CPU get ready to perform "future" calculations more quickly. It gets to guess where the program is going to go, and start fetching resources that it thinks will be needed. As long as nothing architecturally visible is changed by these preparations, everything is functionally the same, so what could go wrong?

And intuitively, you wouldn't think that something like a cache, which the program has no way to access directly, should be architecturally visible. Even putting on a security-minded hat, you would think that it doesn't matter what's in the cache, because if a program tries to access kernel memory, the access still has to undergo a permissions check.

The attack is pretty damn clever. And disheartening.

More or less the conclusions I came to during university. Speculative execution felt dicey (because you're doing things before you know if they are the right things), so I spent some time thinking about it and realised that you are (basically) throwing away or rolling back the erroneous execution. At worst I thought it would be wasted executions if, e.g, your branch predictor is poor - I never even thought about timing or indirect attacks. It's a pretty subtle thing IMO.

IIRC, one of my professors at one point explained modern CPU behaviour (branch prediction/OOOE etc.) as, roughly, the CPU can do whatever it likes because the program never sees what happened under the hood - it just has to order the result/make sure the result is correct.

Reply on news.ycombinator.com