At least we'll always have the PS2. ...until the last one breaks, I guess.
For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent. When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
I realize I've sort of just re-invented Javascript. But what if we just did away with native binaries entirely, except as ephemeral objects that get cached and then thrown away and regenerated when needed? It seems like this would solve a lot of problems. You could deprecate CPU instructions without worrying about breaking backwards compatibility. If a particular instruction has security or data integrity issues, just patch the compiler not to emit that instruction. As new side-channel speculation vulnerabilities are discovered, we can add compiler workarounds whenever possible. If you're a CPU architect and want to add a new instruction for a particular weird use-case, you just have to add it to your design and patch the compiler, and everyone can start using your new instruction right away, even on old software. You'd be able to trust that your old software would at least be compatible with future instruction architectures. Processors would be able to compete directly with each other without regard to vendor-lock-in.
Originally, AS/400 used its own bytecode called MI (or TIMI or OMI). A descendant of the System/38's bytecode. That was compiled to CISC IMPI machine code, and then after the RISC transition to POWER instructions.
However, around the same time as the CISC-to-RISC transition, IBM introduced a new virtual execution environment – ILE (Integrated Language Environment). The original virtual execution environment was called OPM (Original Program Model). ILE came with a new bytecode, W-code aka NMI. While IBM publicly documented the original OPM bytecode, the new W-code bytecode is only available under NDA. OPM programs have their OMI bytecode translated internally to NMI which then in turn is translated to POWER instructions.
The interesting thing about this, is while OMI was originally invented for the System/38, W-code has a quite different heritage. W-code is actually the intermediate representation used by IBM's compilers (VisualAge, XL C, etc). It is fundamentally the same as what IBM compilers use on other platforms such as AIX or Linux, and already existed on AIX before it was ever used on OS/400. There are some OS/400-specific extensions, and it plays a quite more central architectural role in OS/400 than in AIX. But W-code is conceptually equivalent to LLVM IR/bitcode. So here we may see something in common with what Apple does with asking for LLVM bitcode uploads for the App Store.
> And objects contained both the IR and the native code. If you moved a binary to another host CPU, it would be retranslated and run automatically
Not always true. The object contains two sections – the MI bytecode and the native machine code. It is possible to remove the MI bytecode section (that's called removing "observability") leaving only the native machine code section. If you do that, you lose the ability to migrate the software to a new architecture, unless you recompile from source. I think, most people kept observability intact for in-house software, but it was commonly removed in software shipped by IBM and ISVs.
Rosetta 2 AOT, whilst not being exactly the same thing as the ISA agnostic IR solution, is another example of the static binary translation. Theoretically, Apple could start requiring OS X app submissions to the app store in the Bitcode format as well, so they could be transpiled and optimised at the app download time and perform efficiently on M3, M4, M5 etc CPU's in the future. However, with their habit of obsoleting certain things fast, it is not clear whether they will choose to go down the Bitcode path for the OS X apps.
The Mill does something like this, but only for their own chips. "Binaries" are bitcode that's not specialized to any particular Mill CPU, and get run through the "specializer" which knows the real belt width and other properties to make a final CPU-specific version.
Or one of several bytecodes that get JIT or AOT compiled.
WASM in particular has my interest these days, thanks to native browser support and being relatively lean and more friendly towards "native" code, whereas JVM and CLR are fairly heavyweight, and their bytecodes assume you're going to be using a garbage collector (something that e.g. wasmtime manages to avoid.)
Non-web use cases of WASM in practice seem more focused on isolation, sandboxing, and security rather than architecture independence - stuff like "edge computing" - and I haven't read about anyone using it for AOT compilation. But perhaps it has some potential there too?
You've re-invented UCSD P-code.
The BCPL compiler always outputs ocode. Then a back end for the particular system statically generates machine code (or assembly language)
Back in 1983 I helped take a BCPL running on a VAX and add a M6809 back end to it.
Even if we get an ARM RISC-V monoculture, at least we are getting diverse co-processors again, which present the same portability challenges/opportunities in a different guise.
Efforts like Bootstrappable Builds and Debian's rebootstrap aim to reduce the barriers for new architectures:
https://bootstrappable.org/ https://bootstrapping.miraheze.org/wiki/Main_Page https://wiki.debian.org/DebianBootstrap https://wiki.debian.org/HelmutGrohne/rebootstrap
For eg the Synopsis folks are bootstrapping a Debian ARC port right now using rebootstrap.
https://github.com/foss-for-synopsys-dwc-arc-processors/rebo...
IBM's OS/400 (originally for the AS/400 hardware, now branded as System i) did precisely this: Compile COBOL or RPG to a high-level bytecode, which gets compiled to machine code on first run, and save the machine code to disk; thereafter, the machine code is just run, until the bytecode on disk is changed, whereupon it's replaced with newer machine code. IBM was able to transition its customers to a new CPU architecture just by having them move their bytecode (and, possibly, source code) from one machine to another that way.
https://en.wikipedia.org/wiki/IBM_System_i
Other OSes could definitely do it.
See https://en.wikipedia.org/wiki/Architecture_Neutral_Distribut...
Using ANDF you could produce portable binaries that would run on any UNIX system, regardless of CPU architecture. It was never commercially released though. I think while it is cool technology the market demand was never really there. For a software vendor, recompiling to support another UNIX isn't that hard; the real hard bit is all the compatibility testing to make sure the product actually works on the new UNIX. ANDF solved the easy part but did nothing about the hard bit. It possibly would even make things worse, because then customers might have just tried running some app on some other UNIX the vendor has never tested, and then complain when it only half worked.
Standards are always going to have implementation bugs, corner cases, ambiguities, undefined behaviour, feature gaps which force you to rely on proprietary extensions, etc. That's where the "hard bit" of portability comes from.
https://en.wikipedia.org/wiki/Architecture_Neutral_Distribut...
It didn't last.
There are a lot of universal-binary candidates both current and historical. Javascript, Java, SPirV, LLVM intermediate representation are some of the current ones. So, this isn't a new idea, it's just that most of the software I use regularly is compiled specifically for x86-64. Maybe it would be better if that were a rare exception rather than the norm.
NDK's main purpose, from Android team point of view, is for writing native libraries.
> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent.
We're doing a pretty good job on portability these days already. Well-written Unix applications in C/C++ will compile happily for any old ISA and run just the same. Safe high-level languages like JavaScript, Java, and Safe Rust are pretty much ISA-independent by definition, it's 'just' a matter of getting the compilers and runtimes ported across.
Adopting LLVM IR for portable distribution, probably isn't the way forward. I don't see that it adds much compared to compiling from source, and it's not what it's intended for. (LLVM may wish to change the representation in a subsequent major version, for instance.)
For programs which are architecture-sensitive by nature, such as certain parts of kernels, there are no shortcuts. Or, rather, I'm confident the major OSs already use all the practical shortcuts they can think up.
> When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
Source-based package management systems already give us something a lot like this.
There are operating systems that take this approach, such as Inferno. [0] I like this HN comment on Inferno [1]: kernels are the wrong place for 'grand abstractions' of this sort.
> I realize I've sort of just re-invented Javascript
Don't be too harsh on yourself, JavaScript would be a terrible choice as a universal IR!
> [[ to the bulk of your second paragraph ]]
In the Free and Open Source world, we're already free to recompile the whole universe. The major distros do so as compiler technology improves.
> Processors would be able to compete directly with each other without regard to vendor-lock-in.
For most application-level code, we're already there. For example, your Java code will most likely run just as happily on one of Amazon's AArch64 instances as on an AMD64 machine. In the unlikely case you encounter a bug, well, that's pretty much always a risk, no matter which abstractions we use.
[0] https://en.wikipedia.org/wiki/Inferno_(operating_system)
Maybe, but PNaCl (unfortunately deprecated by Google) "defines a low-level stable portable intermediate representation (based on the IR used by the open-source LLVM compiler project) which is used as the wire format instead of x86 or ARM machine code" https://www.chromium.org/nativeclient/pnacl/introduction-to-...
It would be a poor fit for certain languages, there may be performance penalties depending on target platform, it would preclude legitimate platform-specific code such as SIMD assembly, it would preclude platform-specific build-time customization, etc.
The way toward painless portability is to move away from unsafe languages like C and C++, where you're never more than an expression away from undefined behaviour, and where programmers may be tempted to make silly mistakes like writing code sensitive to the endianness of the target architecture. [1] With C and C++, disciplined developers working carefully, can write portable code. With Safe Rust, code can be pretty close to 'portable by construction', like Java. If you feed Windows-style path strings to Linux, or vice versa, then things might go wrong, but for the most part you'll be on solid ground.
[0] https://news.ycombinator.com/item?id=26398199
[1] https://commandcenter.blogspot.com/2012/04/byte-order-fallac...
That sort of works, but duplicating the proper development environment in the end-user's computer would take a lot of space and would be complicated by the enormous variety of programming languages and environments. Linux distros manage this with a lot of effort. I'm imagining something like a universal intermediate representation that can be compiled quickly (because a lot of the early language-specific part of compilation will have already been done by whoever you get your packages from) and in a uniform way because there's a common intermediate representation format that all the compiled languages use.
Universal binaries might also be acceptable for commercial, closed-source applications where source distribution would not.
Most distros offer precompiled binaries, there are relatively few that use source-based distribution and expect the user to have all the necessary compilers installed.
> would be complicated by the enormous variety of programming languages and environments
That problem isn't effectively addressed by a universal IR. You can't have a single IR that works well for all languages, precisely because of the variety of languages.
> Linux distros manage this with a lot of effort.
Hopefully that should improve if the trend toward languages like Safe Rust continues. C and C++ are infamously full of footguns.
> I'm imagining something like a universal intermediate representation that can be compiled quickly (because a lot of the early language-specific part of compilation will have already been done by whoever you get your packages from) and in a uniform way because there's a common intermediate representation format that all the compiled languages use.
Again this can't be done effectively. There are good technical reasons why Java, Haskell, and JavaScript, don't generally use LLVM as their backend. The differences between languages aren't just skin deep, they extend right through the compiler stack.
To be more precise: it could be done, but there would be an unacceptable performance cost. After all, you could start distributing binaries for the SuperH SH-4, and just use emulation everywhere. The question is whether it could be done effectively.
I mentioned before that LLVM IR is not intended to be used this way, although the Google Native Client project took LLVM and turned it into what you're suggesting.
C and C++ are quite different from Java. The size of the int type varies between platforms, for instance. They also have a preprocessor which allows the programmer to conditionally compile platform-specific code, e.g. intrinsics, fragments of assembly code, or workarounds. The program might use system-specific macros that expand before compilation.
Languages like Haskell are very different from the sorts of languages that LLVM is built for. Even Java prefers to use its own backend, with tight integration with its GC.
There's also a package-management question, although this issue wouldn't be as significant. The C/C++ way is to have the build system (autotoools or CMake or whatever) detect what libraries are available on the system. If an optional library is missing, the C/C++ code is automatically adjusted by the build system, prior to compilation. It would be unusual to detect availability of libraries at runtime. This approach doesn't play nicely with a universal IR. This might not be an issue if the IR is treated as a surrogate for the native-code binary, but the IR wouldn't be a good surrogate for the source.
The C/C++ philosophy is to accommodate platform variations, in contrast with the JVM approach of mandating compliance to a virtual machine. With the JVM approach you forbid the sorts of variations that C and C++ permit (everything from int_fast32_t varying between platforms, to hand-written SIMD assembly).
Others have already mentioned WASM and Google Native Client, both of which are stable, but neither of which are going to become mainstream ways of distributing Unix application code.
This topic has turned up on HN before, but frustratingly I wasn't able to find the thread.
> Universal binaries might also be acceptable for commercial, closed-source applications where source distribution would not.
True, but I think modern Unix OSs do a pretty good job on ABI stability. If they want portability without releasing source (something I don't think GNU/Linux should aim to accommodate, incidentally) they already have other options, like Java. JavaFX doesn't get much attention but it pretty much 'just works' for portable GUI applications.
edit skissane has an interesting comment on ANDF, a solution I hadn't heard of before.
What problem did MIPS solve in a unique way that others didn't? Because it wasn't desktop, mobile, embedded, graphics or AI.
After all MIPS's original insight was that machine code was now overwhelmingly written by compilers and not handwritten assembly, so they made an ISA for compilers. I think history proved them absolutely right, actually these days there are often a couple of layers between the code people write and the instructions fed into the CPU.
I guess my point is that nowadays I'm sure that many competent devs don't know what little-endian means and probably wouldn't have any idea of what "register windowing of the stack" is, and they're completely unaffected by these minute low level details.
Making it a bit easier for OpenBSD to find subtle bugs is certainly nice, but that seems like a rather weak argument for the vast amount of work required to support a distinct ISA in a kernel.
Honestly I'm not convinced by the argument for diversity here, as long as the ISA of choice is open source and not patent encumbered or anything like that. Preventing an x86 or ARM monoculture is worth it because you don't want to put all your eggs in Intel or Nvidia's basket, but if anybody is free to do whatever with the ISA I don't really see how that really prevents innovation. It's just a shared framework people can work with.
Who knows, maybe somebody will make a fork of RISC-V with register windows!
These days "bare metal programming" means writing raw HTML yourself in a text editor, no CSS, no JS.
So, I immediately saw it had issues on a variety of different platforms, including an endianess problem. Cpantesters.org lets you drill down and see what went wrong in pretty good detail, so I was able to fix the problems pretty quickly.
It used to have a ton of different platforms like HPUX/PA-RISC, Sun/Sparc, IRIX/MIPS and so on, but the diversity is down pretty far now. Still lots of OS's, but few different CPUs.
Even today, comparing modern MIPS64 and ARM aarch64, I find ARM's new ISA to be perhaps more similar to MIPS than to ARMv7.
https://tvtropes.org/pmwiki/pmwiki.php/Main/SeinfeldIsUnfunn...
The MIPS R2000 was debatably the first commercial RISC chip. It solved whatever problem you needed a really fast CPU for in 1985. The alternatives on the market were the Intel 386 and the Motorola 68000. The Intel 386 at 16 MHz did about 2 MIPS (heh - millions of instructions per second) with 32 bit integer math. At 16 MHz, the R2000 did about 10 MIPS. Even accounting for RISC code bloat, that's 3 - 4x faster.
Note how there were only two competitors selling 32-bit designs in the market they entered. I think that's probably the biggest impact of MIPS. They actually sold the chip! They wanted companies to design their own computer systems around it. Use it in an embedded device. Whatever. That was not the norm c. 1985 - 1988 for high-end silicon.
There were machines faster than the 386 or 68020 at that time. You could buy one of the fast microprocessor-based VAXes recently introduced. Or if not too squeezed for office space and with a blank cheque, one of the super-minis like a real VAX or IBM's new "mini-mainframe". After '86, maybe you'd buy one of the other RISC options, like SPARC or PA-RISC.
Whatever you bought, it would be the whole system. Take it or leave it. DEC would not sell you something like a CVAX processor all by itself just so you can build it into a product that will compete against them. (Well, they would sell you one, just not at a price you could afford if you aren't a defence contractor.)
Both DEC and SGI would use MIPS processors in their workstations of the late 80s, as did some less well-known names. The embarrassment of having to use a competitor's processor to sell a decently fast and affordable UNIX workstation would inspire DEC to create the Alpha. In this vein of "we'll sell it to whoever wants to buy it!" MIPS was also doing ARM-style core IP licensing, before ARM did. That's probably part of why MIPS was so prominent as an embedded architecture in the late 90s and early 2000s, in everything from handhelds to routers to satellites.
I don't know about unique way, but MIPS certainly was good at embedded; there's plenty of networking gear using it.
As far as I understand, it's not that MIPS is the best at embedded, it's just that it's cheaper to sell as the license cost is non-existing and good support already exists in kernels and so on.
If MIPS offered adequate performance and features, good performance-per-watt, and a competitive licence fee, and if none of its competitors could beat it, doesn't that count as 'best'?
That sounds a lot like MIPS being the best at embedded. Not high-end, sure, but a lot of embedded is "what is the cheapest processor that can run Linux?"
I suppose it will live on in that form, especially if the IP is opened up.
I do think the simplicity of MIPS was a big plus there, including simplicity of simulating it (http://spimsimulator.sourceforge.net/). I suppose a lot of students may appreciate being taught on something that is or is about to be very widely used, even if it's more complicated in various ways -- and the fact that one of the textbook authors was a main RISC-V designer makes me assume that educational aspects are not at all neglected in the RISC-V world.
More on topic, though, RISC-V seems to really be designed in a way that makes it easy to teach. This is partially why I have doubts that it can be made very performant, but the focus of a prettier design over a more practical one is probably going to help it be more accessible to students.
Perhaps CPU diversity is in decline but it seems to me that the industry as a whole is moving towards more diversity. It's gotten significantly cheaper to roll your own chips to the point that we've seen entirely new processors emerging (e.g. GPUs, TPUs, etc) and becoming commonplace if not essential.
Isn't the point of RISC-V that the CPU is simple and augmented or complimented by any number of custom co-processors? If this is the general tend in the industry then the CPU itself might become a commodity part to be easily swapped out as something better emerges. Particularly if it can be abstracted away from the ISA.
Patterson says RISC-V is derived from RISC-I and RISC-II. I think this doesn't really old water -- at least no more so than any other RISC.
RISC-I and RISC-II had condition codes and register windows, like SPARC. RISC-V doesn't have either, like MIPS. The RISC-V assembly language is also very similar to MIPS.
RISC-II had both 16 and 32 bit opcodes (as did IBM 801, and Cray designs) and RISC-V has inherited this (but also following the great success of ARM Thumb2).
https://www.amazon.com/Computer-Organization-Design-MIPS-Arc...
https://www.amazon.com/Computer-Organization-Design-RISC-V-A...
I guess as of today we know which one to get.
https://openpowerfoundation.org
https://github.com/antonblanchard/microwatt
Unfortunately it isn't gaining any traction. From a Long term Cost perspective it is actually cheaper choosing ARM even if OpenPOWER is free. And ARM is already inexpensive.
ppc64le and 32-bit Arm are part of the wider family, but have some notable differences that make them slightly less RISC-like. Both include e.g. more complex condition code handling and instructions that operate on more than three registers.
If you want something really different, whether in operating systems or programming languages, you have it. KolibriOS and Haskell, say.