The Intel 80376 – A legacy-free i386 with a twist (2010)
pagetable.com
pagetable.com
1. There are more users of legacy modes than is obvious to us on HN.
2. The gains in terms of gate lost, critical paths reduced, lower power consumption just don't amount to much.
My guess is that #2 is the dominant factor. If there were actually significant gains to be had on a "clean"(er) x86 design, we'd see it in the market regardless of #1.
Dumping legacy features would be great for all kinds of reasons, but not this particular reason.
People get caught up in the variable length issue that x86 has, and then claim that M1 beats x86 because of that. Sure, decoding ARM instructions is easier than x86, but the variable length aspect is handled in the predecode/cache stage, not the actual decoder. The decoder, when it reaches an instruction, already knows various bits of info are.
The RISC vs CISC debate is useless today. M1's big advantage comes from the memory ordering model (and other things)[0], not the instruction format. Apple actually had to create a special mode for the M1 (for Rosetta 2) that enforces the x86 ordering model (TSO with load forwarding), and native performance is slightly worse when doing so.
[0]: https://twitter.com/ErrataRob/status/1331735383193903104
[a]: There's also others that predate AVX (VEX) such as the 0F38 prefix group consisting only of opcodes that have a ModR/M byte and no immediate, and the 0F3A prefix being the same, but with an 8 bit immediate.
Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs).
Related point: Hector Martin just upstreamed a patch to Linux that fixes a memory ordering bug in workqueues that's been around since before Linux had Git history. He also found a bug in some ARM litmus tests that he was using to validate whether or not they were implemented correctly. Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would.
That's not true. (And doesn't your last paragraph contradict it already?)
You might just have figured out that most stuff will run fine (or appear to run fine for a long time) when TSO isn't enabled.
> Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs).
Apple tells you to attach a special volume/FS to your linux VM in order for Rosetta to work. When such a volume is attached, it runs the VM in TSO mode. As simple as that.
The rosetta binary itself doesn't know whether or not TSO is enabled, so its not surprising that it runs fine under Asahi. As marcan42 himself said on twitter[1], most x86 applications will run fine even without TSO enabled. You're liable to run into edge cases in heavily multithreaded code though.
[1]: https://twitter.com/marcan42/status/1534054757421432833
> Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would.
In other words, they're not constantly running in TSO mode? Because if they were, why would they trigger such re-orders?
EDIT: I've just run a modified version of the following test program[2] (removing the references to the tso_enable sysctl which requires an extension), both native and under Rosetta.
Running natively, it fails after ~3500 iterations. Under Rosetta, it completes the entire test successfully.
The rest of my opinion is based off of these three Twitter threads:
https://twitter.com/marcan42/status/1534053625110351872
https://twitter.com/never_released/status/153412764108259328...
In any case, I'm pretty sure that marcan is aware of that, and I haven't read anything from him that would point to the contrary.
And not one of those threads indicates that there is no Rosetta-related instructions. The first would indicate that marcan believes that M1 chips do have TSO related instructions. The second discusses the fact that Rosetta runs on non-Apple CPUs without TSO enabled, meaning that it running on Asahi linux without Asahi linux having TSO support does not show that M1 chips always run in TSO mode; but rather that Rosetta itself has no way of detecting it, so will attempt to run regardless. The third repeats what I just said.
The twitter threads provide more evidence against your points than for them.
What? No. Itanium was a vastly, wildly different architecture.
* IA-64 failed primarily because it failed to deliver the promised performance. x86 comparability isn't and wasn't essential to success (behold the success on Arm for example).
* M1's advantage has almost nothing to do with the weak memory model, but it has to do with everything: wider, deeper, faster (memory). The ISA being Arm64 also help in many ways. The variable length x86 instructions can be dealt with via predecoding, sure to an extent, but that lengthens the pipeline which hurts the branch mispredict penalty, which absolutely matters.
Let's roughly sketch a proposal, even if many variants are possible
Instructions start on 64 bit boundarys. If the highest bit is 0 you have an insn else a constant. So the whole thing becomes self-synchronizing. You're now of course missing 1 bit, that should get a place in the encoding of a constant.
This means you have 62 bits left. Presuming the prefixes like segment regs get solved, you could encode 2 instructions in there, 1 of them a nop if you need the constant. Do away with short encodings for ax. Try and avoid double encodings uf reg + modrm combos exist. Above all, resist the temptation to mess with the instruction set, just re encode.
This could be paired with a sort of two-tiered processor design-- really fast on the restricted subset, but mediocre-to-bad performance on anything else. The code only has to activate the "mediocre to bad" paths long enough to build a JIT cache though,
There are those users who still want to use DOS programs: https://www.freedos.org/
By the way, 64 bit Windows doesn’t run 16 bits applications and never has. The only relevant 16 bit code is InstallShield and that is emulated using trickery. For the rest you need to emulate a whole computer.
https://docs.microsoft.com/en-us/troubleshoot/windows-client...
Sadly, there was a serious design error in the original x86_64 support for i387 floating point, and it was never possible for a 64-bit kernel to correctly context switch some of the more obscure legacy floating point exception state. Rather than fixing it, both AMD and Intel nerfed the hardware support, and new CPUs no longer fully support this state. This makes some older 16-bit software work poorly.
Whether the instructions come from the kernel or from userspace does not matter at all, they all go through the same decoder and functional units. The kernel/userspace differentiation is a higher level concept.
Long story short, this isn't really the bottle neck, or we'd see more simple decoders on the tail end of the decode window.
Plus I'm sure there's crazy DRM rootkits that depend on these implementation details.
Also, AMD has experimented with not-quite-PC-compatible x86 setups already in the consoles. As the fail0verflow talk about Linux on PS4 emphasised, the PS4 is x86, but not a PC. So despite building a x86 CPU with some less legacy, AMD didn't seem to think it worthwhile bringing it to a more general purpose platform
Also AMD/Intel/VIA are the only companies with the licenses to produce x86, and you'd need both Intel and AMD to sign off on licensing x64 to someone new.
Except that a modern OS is booted in UEFI mode meaning that the steps of going fro 16 -> 32 -> 64-bit mode are all handled by the firmware, not the kernel or the bootloader. The OS kernel will only (at most) switch to 32-bit compatibility mode (a submode of long-mode not protected mode) when it needs to run 32-bit apps, otherwise staying in 64-bit mode 100% of the time.
That may be only secondary, though. Itanium simply failed to deliver performance promises and be competitive. The compiler was supposed to effectively perform instruction scheduling itself, and writing such a compiler turned out more difficult than anticipated.
OoO EPIC has been suggested (I recall an old comp.arch posting by an Intel architect) but never got green-lit. I assume they had bet so much on compiler assumption that the complexity would have killed it.
It's really a shame because EPIC did get _some_ things right. The compiler absolutely can make the front-end life easier by making dependences more explicit (though I would do it differently) and by making control transfers much easier to deal with (the 128-bit block alone saves 4 bits in all BTB entries, etc). On the balance, IA-64 was a committee-designed train wreck, piling on way too much complexity, and failed both as a brainiac and speed-daemon.
Disclaimer: I have an Itanic space heater than I occasionally boot up for the chuckle - and then shuts down before the hearing damage gets permanent.
EPIC’s problem is shared with VLIW of which EPIC can be understood as a refinement. VLIW excels in a deterministic world where the compiler can predict latencies and produce a good schedule, but falls apart in face of loads with highly variable latencies (an in-order implementation has no option but to stall when load data doesn’t arrive on time).
EPIC patches this a bit by allowing software prefetching and SW exposed speculation, but it comes at a significant code bloat and it can cover only a small fraction of what dynamic scheduling can cover. At Transmeta, our VLIW engine did this too (with some obvious advantages over IA-64) and we suffered from similar problems.
Micro-architecture is still a hot topic and everything is a trade-off. Next I’ll start talking about super scalar OoO stack machines …
More than once. iAPX432 if anything went worse.
As an aside I’m still astonished that 432 and Itanium got as far as they did with so much cash spent on them before without conclusive proof that performance would be competitive. Seems like a prerequisite for projects of this size.
The Cortex-A78 generation of Arm cores was the first set of Arm-designed cores that didn't support AArch32 at EL3, and they retain AArch32 support at EL0, even the N1 and X1. The early APM designed ARMv8 cores supported AArch32 at (at least) EL0, although I haven't kept up with that lineage. I haven't seen an nVidia-designed ARMv8 that dropped AArch32, although I also haven't worked with some of the newer ones.
I suspect that maintaining support at EL0 is very cheap, because it can be done entirely in the instruction decoder.
Qualcomm dropped Arm32 in Falkor, which shipped in 2017: "AArch64 only" https://en.wikichip.org/wiki/qualcomm/microarchitectures/fal...
Arm64 is really a wild and radical departure from Arm32, they're night-and-day different which is why dumping 32-bit support is attractive. Compare this with x86 where you have a microcode backdoor^H^H^Hengine to smooth over the differences, or POWER64/MIPS64 which built off of very clean 32-bit designs and stayed true to them. And of course RISC-V is the non-trademark-infringing name for MIPS-VI.
Wild speculative guess: It's entirely possible some of the pre-AMD64 stuff is actually internally used by modern Intel and AMD CPUs to implement complex instructions.
Jokes aside, I doubt XuCode would use pre-AMD64 stuff; microcode is lower-level than that. The pre-AMD64 stuff is already handled with sequences of microcode operations because it's not really useful for modern applications[0]. It's entirely possible for microcode to implement other instructions too, and that's what XuCode is doing[1].
The real jank is probably hiding in early boot and SMM, because you need to both jump to modern execution modes for client or server machines and downgrade to BIOS compatibility for all those industrial deployments that want to run ancient software and OSes on modern machines.
[0] The last time I heard someone even talk about x86 segmentation, it was as part of enforcing the Native Client inner sandbox.
[1] Hell, there's no particular reason why you can't have a dual-mode CPU with separate decoders for ARM and x86 ISAs. As far as I'm aware, however, such a thing does not exist... though evidently at one point AMD was intending on shipping Ryzen CPUs with ARM decoders in them.
The fact that a cleaned version wasn’t used would seem to support your hypothesis.
> than is obvious to us on HN
I think your average HNer is more likely to interact with Linux/Mac workstations or servers, where binary compatibility isn't as necessary.
The reason is the PC world at large is a mountain of x86 and Win32 code piled sky high by decades of developers and users. Replacing all that takes money and time that nobody wants to spend, and chances are the replacement won't be a 1:1 drop-in replacement either which will necessitate further money and time nobody wants to spend.
The average computer enthusiast might scoff at being able to run half-a-century old x86/Win32 code and binaries on modern hardware. Professionals consider that backwards compatibility their literal lifeblood and will pay handsomely to ensure nobody violates it (and refuse to pay anyone who does).
I expect the ARM world will also be singing a similar tune in several decades' time, when backwards compatibility with ancient ARM binaries become steadily more important than pursuing the next new shiny at any cost.
The various Rosettas, most recently the x86-to-ARM one, showed that software emulation is always an option, and so does WoW64, which is a part of ARM Windows (and a part of x86-64 Windows, but there it's not really a software emulator).
> I expect the ARM world will also be singing a similar tune in several decades' time, when backwards compatibility with ancient ARM binaries become steadily more important than pursuing the next new shiny at any cost.
I am not so sure at all about that. The reason is that half to a quarter decade ago, even normal application code was very close to the metal, with MS-DOS basically being a glorified bootloader with file system services that loaded little mini-OSes calling themselves DOS applications.
We are far, far away from that world. Mature abstractions make ISA transitions more and more painless for end-user application developers. The exponential advances in speed make software emulation well feasible for the remainder. High-performance stuff will want to take advantage of the new architectures anyway.
Half to a quarter century, I mistyped here. Half to a quarter decade ago we were long since there already.
More importantly, though, backward compat has been Intel's moat for a long, long time. Intel have been trying to get people to buy non-16-bit-compat processors for literally 40 years! They've tried introducing a lot of mildly (StrongARM - technically a buy I suppose, i860/i960) and radically (i432, Itanium) innovative processors, and they've all been indifferent or outright failures in the marketplace.
The market has been really clear on this: it doesn't care for Intel graphics cards or SSDs or memory, it hates non-x86 Intel processors. Intel stays in business by shipping 16-bit-compatible processors.
In many segments, binary compatibility with legacy software is still vital. This is why you can run Windows 3 software on Windows 11 (and why you can run x86 software on M1 Macs).
For me, I could spend most of my time on Linux and there binary compatibility is a much smaller issue - I'm sure I'd be very happy with a POWER workstation (birthday is in March, Raptor folks) and perfectly functional there. Same on ARM (biggest constraint for me is memory - 16 GB is enough, 32 is good).
Unless I look too closely, the CPU in the box is the least important part.
Seems like a pretty good case for the counterargument: that backwards compatibility can be entirely provided for in software.
(To be clear, the M1 chip doesn't support the x86 instruction set at all, and translation is done entirely in software. I don't think there was ever any 16 bit x86 Mac software though, so not sure if that mode is supported).
I don't think Macs ever had 16-bit code? 68000 was 32-bit, as was ppc and intel macs were 32-bit to start, with 64-bit later.
Yeah pretty sure you are right.
> 68000 was 32-bit,
Interestingly I discovered today the 68008 which was a 68000 with an 8-bit external data bus.
Later variants of the 68k family were fully 32-bit internally, but exactly what was exposed externally varied by processor and Mac model. And 24-bit addressing remained a thing in the Macintosh system software up until System 7-- this capped RAM at 8 MB even if the CPU could handle more.
But Apple never really had the same sort of 16-bit/32-bit transition that Microsoft and Intel did. The 68k-to-PPC shift is kind of analogous (and happened at around the same time as Microsoft was going fully 32-bit), but backwards compatibility there was purely based around software emulation-- same basic approach as Apple's PPC-to-x86 and x64-to-ARM transitions, as opposed to Intel's "just make the new processor a superset of the old one" approach.
Chip size and pin count. Besides, 24 address lines allowed 16 megabytes of RAM, something outrageous back then.
My point was that it's so vital on some platforms that their vendors go to great lengths to make it available, despite the fact that the architecture is completely different.
I don't see the same requirement in open-source stacks.
So, hypothetically, at least in the Windows world, we're approaching a point where some of the legacy cruft could go. Not quite all of it yet (still need 32-bit application support), but a lot of the 8086-era real-mode stuff will never be used on modern hardware running modern software (except perhaps for the first few instructions after boot, until UEFI pushes the system into 32- or 64-bit protected mode).
Apple's proven it feasible to provide high performance emulation of a prior ISA.
Wait - what? How is that even possible? Do they simply not have an MMU? That makes it unsuitable for both old OSes and for new OSes. No wonder it was so uncommon.
Intel CPUs before the 80286 did not have an MMU, either.
So effectively the same result as an MMU. But Minix is the only OS I know of which used this mode.
If an MMU is something that sits between memory and the rest of the CPU, and abstracts physical addressing away, then I argue the 286 did not have an MMU. Segmented access is more of a mandatory addressing mode, where you always access relatively to a segment's base, and have to keep within the limit.
OS/2 is another OS that made use of pre-paging 286 protected mode.
"The 286 was the first of the x86 CPU family to support protected virtual-address mode, commonly called "protected mode". In addition, it was the first commercially available microprocessor with on-chip MMU capabilities (systems using the contemporaneous Motorola 68010 and NS320xx could be equipped with an optional MMU controller). This would allow IBM compatibles to have advanced multitasking OSes for the first time and compete in the Unix-dominated[citation needed] server/workstation market."
That Wikipedia passage seems to be inaccurate, or a least not use the common terminology.
""In Protected Virtual Address Mode, the 80286 provides an advanced architecture that retains substantial compatibility with the 8086 and other processors in the 8086 family.""
""In Protected Mode, application programs deal exclusively with virtual addresses; programs have no access whatsoever to the actual physical addresses generated by the processor. As discussed in Chapter 2, an address is specified by a program in terms of two components: (1) a 16-bit effective address offset that determines the displacement, in bytes, of a location within a segment; and (2) a 16-bit segment selector that uniquely references a particular segment.""
I'm still not sure if I like calling that an "MMU". To me, an MMU acts on every memory access. But at the end of the day, who cares what I think what words mean. :)
The critical distinction to me is that segmentation is not much more than, effectively, an indexed addressing mode, where the base is the segment base that has been loaded into the descriptor cache when the segment register was loaded. Nothing happens at the time of actually accessing memory, except for the limit check.
This is not correct.
Intel referred to the 286's memory management as virtual memory:
Quote from the 286's manual - 4th paragraph of section 1.1
> Segmentation also lends itself to efficient implementation of sophisticated memory management, virtual memory, and memory protection.
The 286 protected-mode segmentation is dramatically different from the segmentation on the 8086.
Virtual memory is merely the ability to use logical addresses that don't necessarily have a direct relationship with a physical address. Your OS can relocate data in physical memory without an application having to know.
On the 286, this is facilitated via the Global and Local Descriptor tables. When you loaded a segment register, it would first check whether your current permission level could access that segment. Exception if you couldn't. Then it would check if the segment was marked as present. Exception if it wasn't present. Then it would load the base address (a full 24 bit physical address) and the size into internal, hidden registers. The hardware would bounds check accesses to segments.
The GDT contained segments that were common to the whole OS, and the LDT contained "process" specific segments.
Intel advertised the 286 as having 1 GiB of virtual address space per process - up to 8K entries in GDT, up to 8K entries in LDT, up to a 64K segment size 16K * 64K = 1G.
Some examples of virtual memory behaviors:
A process tries to allocate more memory but the system is out of physical memory. The system decides to swap another process's memory to disk to make room. The data from the other process is written to disk, the descriptor table entry is marked as not present, and the physical memory is free'd back to the OS. The new process's allocation is placed there. When the old process switches back in, accesses throw a segment-not-present exception and the OS must retrieve the segment from disk. It finds physical memory to back the segment, copies the data from disk, updates the base address in the descriptor table, and reloads the segment register. As far as the process is concerned, nothing has changed. The pointer values are still all the same yet the physical address is different.
On a modern processor, this would be done on a page by page basis, on the 286, you had to do it a whole segment at a time. If your segment's size is 256 bytes, awesome, if it was 64 KiB, that sucks. You also can't punch holes in the middle of segments.
Another example:
Your process has a heap from which it performs dynamic allocations. This heap is pointed to by a descriptor in your LDT. You want to make another allocation but you've exhausted your heap. You call into the OS to grow the size of the heap. Unfortunately, a different object was placed in physical memory right after your heap, so, you find a new spot in memory large enough for the new heap. You copy the contents of the heap to the new address and update the descriptor. Again, the program is none the wiser.
This is a major weak point for the 286. On a modern processor, you'd just map more pages in because with a paged MMU, contiguous virtual addresses don't have to be physically contiguous.
Obviously, this is very different to today's demand paged MMUs (well, the last 37 years anyhow) and it's not nearly as flexible, but it's still virtual memory.
That's not even getting into the neat hardware multitasking thing that automated switching all these registers around (which existed on the 386 but was never used because it assumed you'd still want to use segmentation and spent time and memory backing up registers and doing permission checks most software didn't want).
IMHO - the only reason the 286 is merely a footnote in history is because Intel didn't provide a way to cleanly switch from protected mode back into real mode. Intel erroneously assumed applications and operating systems would just be rewritten for the 286's native mode. It was a huge step forward from the 8086 but the largest userbase of Intel CPUs (DOS users) never saw the improvements beyond a simple speed boost.
The main reason this wasn't used more is that is isn't a good fit for what an OS needs to do.
Jumping to a TSS or task gate saves the current context and restores that for the new task, in one atomic operation. But most kernels save the registers on syscall/interrupt entry (while remaining in the context of the current task), and restore a possibly different context on exit when the current one gets suspended.
There is no way to do "half of a task switch" with the TSS mechanism, so it would always do useless work:
kernel code does:
push user registers
handle syscall / interrupt
select new task and jump to it
CPU does:
save context A (mostly garbage values at this point)
restore context B (containing previously saved garbage)
kernel code does:
pop user registers
return to ring 3In a paged MMU, it's more apt to say that it sits "between" CPU and memory. The frame in physical memory to be accessed is different for every page, and faults can happen while resolving that page. It happens during access, not upfront.
I concede that Intel already called this "virtual memory", I searched in my 386 manual wrong.
:)
http://web.archive.org/web/20220816012342/https://www.pageta...