"In practice, CPUs supporting out-of-order execution support running operations speculatively to the extent that the processor’s out-of-order logic processes instructions before the CPU is certain whether the instruction will be needed and committed. In this paper, we refer to speculative execution in a more restricted meaning, where it refers to an instruction sequence following a branch, and use the term out-of-order execution to refer to any way of getting an operation executed before the processor has committed the results of all prior instructions."
In this explanation, the author starts by showing two different code branches, which is misleading. Meltdown does not require code branches - which is what makes it so surprising. This is the C code example from the paper:
raise_exception();
// the line below is never reached
access(probe_array[data * 4096]);
No branches: you have an exception, and then in the code following that exception, you have some memory access. Despite the exception, the access happens because of out-of-order execution. The actual exploit is, in assembly: ; rcx = kernel address
; rbx = probe array
retry:
mov al, byte [rcx]
shl rax, 0xc
jz retry
mov rbx, qword [rbx + rax]
The exception is raised on the mov command, as it loads a kernel address. This exception will eventually cause the processor to abandon all of the current code it is executing, and the program will terminate from a segmentation fault. But. There is a race condition: before the processor deals with the exception, but after the memory has been accessed, the second mov instruction executes, which uses the data which caused the exception. This shouldn't matter, as execution is abandoned, but data is brought into the cache based on this value, and using side-channel attacks, we can figure out what this value was. From the paper:"To load data from the main memory into a register, the data in the main memory is referenced using a virtual address. In parallel to translating a virtual address into a physical address, the CPU also checks the permission bits of the virtual address, i.e., whether this virtual address is user accessible or only accessible by the kernel. As already discussed in Section 2.2, this hardware-based isolation through a permission bit is considered secure and recommended by the hardware vendors. Hence, modern operating systems always map the entire kernel into the virtual address space of every user process.
As a consequence, all kernel addresses lead to a valid physical address when translating them, and the CPU can access the content of such addresses. The only difference to accessing a user space address is that the CPU raises an exception as the current permission level does not allow to access such an address. Hence, the user space cannot simply read the contents of such an address. However, Meltdown exploits the out-of-order execution of modern CPUs, which still executes instructions in the small time window between the illegal memory access and the raising of the exception."
I find the paper to be very readable. They give a good overview of modern computer architecture, and then walk through all of the steps of their attack. I highly recommend reading it: https://meltdownattack.com/meltdown.pdf