Speculating the entire x86-64 instruction set in seconds with one weird trick
blog.can.ac
blog.can.ac
At the risk of unwarranted self-promotion: the other side of this equation is fidelity in software instruction set decoders. x86's massive size and layers of historical complexity make it among the most difficult instruction formats to accurately decode; I've spent a good part of the last two years working on a fuzzer that's discovered thousands of bugs in various popular x86 decoders[2][3].
[1]: https://github.com/xoreaxeaxeax/sandsifter
[2]: https://github.com/trailofbits/mishegos
[3]: https://ww.easychair.org/publications/preprint_download/1LHr
Once upon a time, reposts were allowed, and so if something was topical and popular, sometimes you'd see multiple items on the front page pointing to the same URL. Sometimes the second or third posting would get more karma simply because of an editorialized title. This was undesirable, so they added the repost merging.
Later, an emergent behavior was that posts that report on something found on another site and had been previously posted would find their way to the front page. Some subset of HN visitors would be newer than the original post, and simply upvote something they hadn't seen before. But also there would be times where the post would be interesting with the added value of hindsight, or would provide context to the topic-of-the-day. So at some point it was decided that reposts would be allowed in the long-term, since sometimes they had value.
Thus we're at the state we have today.
He found 13 likely candidates for unpublished ops, including the 2 that were recently found. Also a few unpublished quirks of some known instructions.
The CPU tries very hard to guess which way branches go and that allows it to speculate much further than if it tried every possible combinations.
What happens in this post is that the author writes a CALL instruction, but then manipulates the stack so that it doesn't actually return where the CPU expects it to. So the CPU will speculatively execute the instructions that follow the CALL linearly, even though they are never actually reached!
Now someone needs to do a follow-up to see what software uses these hidden opcodes :)
At this point sunsetting x86 is not even a pipe dream, most people carry ARM powered computers in their pockets and Apple recently demonstrated that it can be quite successful in high performance devices.
Moving everything to ARM will not be cheap. As for bugs that is entirely dependent on the company making the chip, which ones do you have in mind? (Also recall that M1 is vulnerable to Spectre too).
I kind of hope X86s days are numbered as well, but I'm not looking forward to an ARM monoculture.
I thought the same thing until I saw how well Apple's Rosetta 2 works. Now that we've seen what's possible, I'm hoping that the x64 emulation in Windows/ARM will rise to the challenge set by Apple.
The memory model itself isn't patentable; the TSO memory model already existed on x86, as well as many other architectures before that. Having an option to enable/disable TSO might be patentable, but it'd be a stretch; there's plenty of precedent for allowing a CPU to select between operating modes at runtime. (For example, many PowerPC parts could be switched between little-endian and big-endian modes at runtime, and some developers even used this to assist in emulating x86!)
What's more likely is simply that other ARM SoC vendors aren't implementing their own cores (e.g. they're using a standard Cortex-Ax design from ARM), so they can't add deeply integrated features like TSO themselves.
PowerPC's biendianness would be patentable too if it had been an absolutely ancient technique about as old as computers themselves.
And there's people other than ARM and Apple making ARM cores. Samsung's Exynos M5 should be coming out before too long for instance.
It does not.
edit: this patent from 2012 from IBM references SAO:
And it'd be incredibly stupid of them not to have a patent on it unless there's some clear prior art for exactly their implementation since we're in a first to file system in the US. Otherwise they're asking for someone else to patent it and troll them for an easy cash grab.
See "5.2.1.4 PSTATE_mem_model (MM)" in https://cr.yp.to/2005-590/sparcv9.pdf
Hence my original statement about the "cute _optional_ TSO memory model".
And the runtime isn't a security boundary so there's no reason to hide it.
I also don't see how what you're saying requires hardware support.
You can see how the build directories for qemu are {architecture}-softmmu for the full system emulators, and {architecture}-linux-user for the user mode emulators for another example of this but with Linux binary emulation.
Can you elaborate on this? I was having a look at the M1 instructions here [1] and it seems that they implement at least one of the barriers, CSDB, which is actually documented by ARM[2].
[1] https://dougallj.github.io/applecpu/firestorm-int.html
[2] https://developer.arm.com/documentation/ddi0596/2020-12/Base...
E) All of the above
No, really, Apple are going out of their way not to document certain details about the M1, and this is almost certainly the reason. ARM are scared to death of fragmentation, Apple somehow managed to get away with doing it anyway, and ARM do not want anyone else to get any ideas.
ARM probably has some trademark mechanisms that they could in some world use to enforce compliance from Apple, but Apple has been super duper careful to refer to it as Apple Silicon rather than ARM.
How do you know? ARM's license agreements with Apple are confidential and just about nobody knows what's actually in them (and the few who have seen them aren't allowed to talk about it.)
The standard terms which ARM offers to its other customers may be quite different from the terms it has negotiated with Apple. Apple is a customer with unique clout and leverage, which means they can extract terms in negotiations which other customers may not be able to.
If ARM offers Apple (or any other customer) "special terms", it wants to keep that fact confidential, so that other customers who have the (less generous) standard terms don't start pressuring for special terms for them as well.
These instructions enable an entire new set of parallel execution modes, among other things, to run more protected kernel code. Completely nonstandard stuff.
Also, Apple somehow managed to push custom kernel support to the documentation of kmutil, but not the actual binary, until several macOS releases later. That would only happen if the latter got pulled for some extraneous reason.
I've been working on this since December; the entire situation is all showing signs pointing straight at ARM's lawyers having gotten involved to some extent.
The bring up stuff could be explained by being scared about some patent nonsense.
It is: "They are following ARM's special rules for Apple only but they aren't allowed to tell anyone that those special rules for Apple only exist"
They might decide to leave certain processor extensions undocumented, even close source code which uses them, because they are worried disclosing them would implicitly reveal the existence of the "special rules for Apple", possibly violating the confidentiality clauses of the ARM-Apple contract containing those special rules.
So I still don't see how you know that "There is no such 'do wtf you want' license", when the behaviour you describe is compatible with their being one, but its existence being secret
The problem is that the M1 threw a wrench in the works because now suddenly running non-Apple code that can use these instructions is allowed, and everyone else can see that they exist, so suddenly the existence of that special exemption is public.
I agree except I'd say that it is part of ARM's official license offering, but only if your name is "Apple".
I have to say there is a "do wtf you want" license.
Hear me out. Say you like to speed and you have the money to pay fines and were okay with that. To a person like that what most people see as a "fine" becomes a "tax/fee."
What am I getting at? Apples probably has a hand full of multi-layered legal strategies that ARMs legal team is somewhat aware of. Apple has the money to throw at it. Heck, they can start investing any estimated payout and if they do lose they will of still profited the interest from that investment plus the increased sales they think they are going to get.
Which may mean that ARM wants them to be quiet about. Not because Apple is breaking the rules, but because ARM doesn't want to rub into its other customers the fact that Apple has a special deal that other customers aren't being offered.
That makes it sound like it is some kind of "tacit understanding", "I won't say anything if you don't say anything". I'm suggesting that maybe Apple's license agreement explicitly states that they are allowed to do all this, but also maybe its confidentiality clauses prohibit either party from publicly acknowledging that fact without the other party's explicit permission.
A company like Apple – who has a well-funded legal department, and I've never heard any suggestion that they are anything other than competent – wouldn't set up a multi-billion dollar business on the foundation of "agreed to look the other way". They'd have it all set out in writing, crystal clear and totally secret.
Now they can, since the M1 is an open platform (running third-party kernel code is supported).
https://github.com/AsahiLinux/docs/wiki/HW:Apple-Instruction...
Plus their funky intel-emulation related CPU features which introduce architectural EL0 state (SSE-specific FP flags, AP flags). Plus their hardcoded VHE=1 spec breakage now becomes relevant at EL2. And almost certainly more things we haven't figured out yet.
I don't know much about how stuff is implemented these days, but virtual 8086 mode involves some page table muckery and similar. Surely implementing it creates a larger exposure area, from a security standpoint.
Unfortunately I don't have a better source for this than an anecdote.