I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all.
I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all.
Well no, it would be more accurate to say "a noteworthy example of when you release a chip that isn't in line with what common programming languages can do".
That is, there is nothing wrong with doing the above. There's something wrong with doing the above when you most people are writing sequential C++ :)
There simply wasn't enough ILP to extract :)
IE no amount of compiler advance would have saved it.
A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly.
This is not possible to do as well statically. You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough.
In a lot of workloads, most time is spent stalling for memory.
Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline. On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy.
You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance.
There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on.
TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
Whole program compiler can insert JIT-calling code (specializing for common pattern), JIT compiler cannot view whole program.
One of the things JITs can't do is to add and/or remove parallelism from the program (i.e., add/remove OpenMP pragmas).
Of course they can. A trivial practical example is the JVM JIT using vector instructions to add parallelism to a program that didn't have any before. But a JIT compiler could also profile code and add thread-level parallelism if it wanted.
FM is NP-hard. Omega test is imprecise.
And in general you need more complicated checks like aliasing checks. Which are equal to stopping problem, most of the time.
Usually you can't spend that amount of time on EACH INVOCATION of a program.
https://en.wikipedia.org/wiki/Speculative_multithreading
You can use hardware functionality such as TSX to support this.
It is unbelievable you even suggested it.
TSX won't save you.
Why do you think that? What do you think the optimistic software transactional memory algorithms are doing if not speculating on parallel access to shared memory?
Because Microsoft tried to add STM to the .Net and get 4x slower code after extensive effort: http://www.mail-archive.com/haskell-cafe@haskell.org/msg7973...
Should I continue from these two arguments?
You may see it as a proof of your point, but I see it as a counterexample - a team of smart people was not able make speculative parallelism efficient even with static analysis. JIT will be even less efficient as it has to keep and account for tracing information and for speculation logs. These two interact in interesting way, I guess.
The first stage is the LLVM front end, Clang and other language parsers. They generate IR.
The middle stage is the LLVM optimizer, opt, which takes IR and runs a variety of optimization passes. Opt takes and gives IR.
http://llvm.org/docs/Passes.html
The third is the backend code generator. It lowers IR to the target ISA. There's a tablegen description, there's are MI scheduler models which abstract architectural information as well.
http://llvm.org/docs/WritingAnLLVMBackend.html
I think you really meant can you write an optimizing pass specifically for your target? Yes. But I don't think that modifying the real front end Clang is going to get you much of anything. Intrinsics are different; that's the programmer inserting ISA specific instructions rather than Clang extracting ISA specific information. Really, modifying Clang isn't going to get you anything.
Thomas, what were the 'small' Clang specific tweaks?
No, porting GCC to a new architecture is not a big deal. The really complex part is the ABI, and what I have heard from former GCC developers is that it's as much of a mess in LLVM, because it's simply a messy domain.
LLVM also has a bunch of awesome projects like Poly for polyhedral compilation that we are able to use that don't exist at all for GCC, or are very difficult to integrate. LLVMs lego block like architecture (it was originally designed as a tool to help build toolchains) plus things like Tablegen enabled our single "real" software engineer/CTO to do initial bring up of a backend in under a month.
The problem with Itanium is that it wasn't OoO and can't be made such.
Just to give you a hint - just computing various kinds of addresses in advance can make huge difference. Compute branch address, issue a fetch, compute load address, issue a fetch (even when destination register is blocked). Voila! Your program is sped up significantly.
It wasn't as effective as OoO in the end, but not for lack of attention to the problem.
With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation. Combined with the low clock speed of the KNL, you really don't want to be doing scalar operations if you care at all about performance. Of course SIMD isn't the same as VLIW, since the operation you're doing has to be the same in all vector lanes, but with masking support in AVX512, it is getting a little closer to that.
Also compilers have gotten quite a bit smarter and with hardware die shrinks taking longer, I wouldn't be at all surprised if VLIW ISAs were to make a comeback. I guess the Itanium was a bit ahead of its time.
What recent advances have made VLIW any more tractable than trace scheduling already did? BTW, trace scheduling works for scheduling superscalar processors as well.
VLIW never went away. It's used in embedded, the TriMedia processors. It's used in the REX Computing NEO chip. The Mill CPU is a VLIW of sorts.
I'm not anti-VLIW but I don't know of any recent breakthroughs that make it any more tractable now for non-embedded, non-HPC general purpose computing.
[1] Trace Scheduling: A Technique for Global Microcode Compaction
https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d3...
[2] Bulldog: a compiler for VLIW architectures
VLIW has gotten an extremely bad rap outside of the embedded space due to Itanium, which I strongly contend was not a VLIW in spirit (which you cans see me talk in more depth on here: https://youtu.be/ki6jVXZM2XU?t=441). Itanium introduced a ton of indeterminism by having static scheduling unfriendly things like branch prediction, variable latency caches, and still trying to have some level of support for x86. The core problem for Itanium is that it was impossible for the compiler to make good decisions in the vast majority of applications since the compiler could not know exactly where and when things in memory would be. I believe the underlying failure point has been computer architects for failed VLIW architectures not realizing that fancy bells and whistles and "cleverly complex" hardware designs is the exact opposite of what you want when the compiler needs to be able to do purely static allocation. Adding fancy dynamic pieces in hardware to account from this (seen in a lot of places with Itanium) just makes the problem worse.
The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip, which actually gives the compiler the information it needs to make good decisions and very condensed code. I think the jury is out on the applicability for "general purpose" code on VLIWs... it is not something we at REX really care about at the moment, and I think RISC-V is a great solution with improvements over x86 and ARM.
These are the same reasons why DSPs have been VLIW since the 90s, and our initial applications overlap a lot. It has been sad to see the trend of TI and other DSP makers going down the path of adding hardware caching (that reduces determinism and energy efficiency) to try to "make it easier for programmers" when they already have a decent enough architecture, but a lack of good compiler tools to take advantage of it.
I quite like RISC-V and think it's elegant and simple, but maybe I'm wrong :)
As was pointed out in the A History of Modern 64-bit Computing cite, both RISC and Itanium show an academic bias. Yes, RISC-V is elegant and simple and that helps in a one semester upper div architecture class. But that doesn't help you much outside of a one semester upper div architecture class.
So what do you tangibly get with that elegance and simplicity? You aren't going to write RISC-V assembly; compilers are better especially for such a regular and spare ISA. This is 2017 and LLVM has been a thing for quite some time. Compilers are not a big deal.
Lower power? Chip area? How much lower power and chip area are we talking about?
That being said, greater efficiency is the main point, it's not too difficult to get a RISC-V core that beats (albeit marginally) an equivalent ARM core.
Optimizing compilers can often be difficult though, especially if you want to use novel modifications to the backend to make things more efficient.
Read up on horizontal microprogramming. A RISC ISA is an encoding with an opcode. A μop is really just a long set of fields with some bits clocking some FUs on and others off. There is no decoding; there is no opcode.
A complete macroinstruction is executed by generating an appropriately timed sequence of groups of control signals (micro-operations).
http://www.frsn.utn.edu.ar/tecnicas3/manuales/Microprogrammi...
Originally, RISC cpus were hardwired but they are general purpose ISAs. RISC-V is fixed 32b ISA but has multiple formats controlled by an opcode.
https://image.slidesharecdn.com/20160130deckyfosdemportinghe...
Original microprogrammed CPUs had a writeable control store (WCS). Modern processors translate CISC instructions into μops and cache this translation. The difference is the μop cache (μstore) vs WCS. FWIW, Skylake also has a WCS for obscure instructions.
Microinstructions, μops, do not have an opcode. They are not decoded. Bits in a horizontal μop gate logic on/off directly.
The RISC-V people do say RISC-like.
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-...
And even that's wrong. But then they never even define what a RISC is. So RISC-like could be anything.
As for upstreaming, it is something that we want to do, but are not sure if it will be accepted as our chip is not generally available at the moment. It would probably be done around the same time as our next tape out (which would be in larger quantity than our first one) when we would have generally available development kits this coming winter.
I have been working on public documentation in my spare time... originally supposed to go out in March but we have had other priorities. If you watch my Stanford talk I show the basic ISA overview as well as the WIP documentation made using doxygen. I'm hoping to get stuff out on our website before our next media push in ~June.
This is a fascinating idea, but I have a question.
What happens when the next process generation lets you improve cache timings? What about newer, faster DRAM (or whatever) timings?
Do I have to recompile the world in that case, to see any improvement in performance?
First, your silicon will be smaller - no need for pipelines, even double precision floating point multiply can be done in one clock cycle. Less registers, less silicon area. You can internally do that operation at higher frequencies than main core, timesharing die area for SIMD operations - saving silicon in other way.
Second, your compiler will be simpler - no need for NP-hard algorithms to schedule operations in the basic block, no need for fancy software pipelining for loop code to be warmed up and/or cooled down.
Third, your CPU won't be faster than memory. Your code won't get big hit from missed cache access. And cache will be used to save energy, not the time.
That's exactly the reasons behind one of exascale CPU architectures. It was long ago - circa 2012. I remember they managed to get FLOP/W ratios much higher than Intel's offering at the time.
I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.
But yes, outside of carefully tuned kernels (e.g. GEMM and FFT), you're not going to see a 2x speedup. But that's no different from many other ISA extensions.
In-order processors also leave instruction scheduling to the compiler. They're very common. The workhorse ARM Cortex-A7 is in-order, dual issue and pipelined.
IE if everyone is writing sequential C++, making a chip that is really good for parallel fortran is .... silly.
No amount of compiler advance would have saved itanium. Research early on shows except for very contrived kernels, there wasn't enough ILP to extract.
It actually did a really good job of doing what it could :)
Even the most "sufficiently advanced" compiler can't parallelise everything.
“Davidson also pointed out two areas where academic research could create a blind spot for architecture developers. First, most contemporary academic research ignored CISC architectures, in part due to the appeal of RISC as an architecture that could be taught in a semester-long course. Since graduate students feed the research pipeline, their initial areas of learning frequently define the future research agenda, which remained focused on RISC. Second, VLIW research tended to be driven by instruction traces generated from scientific or numerical applications. These traces are different in two key ways from the average system-wide non-scientific trace: the numerical traces often have more consistent sequential memory access patterns, and the numerical traces often reflect a greater degree of instruction-level parallelism (ILP). Assuming these traces were typical could lead architecture designers to optimize for cases found more rarely in commercial computing workloads. Fred Weber echoed this latter point in a phone interview. Bhandarkar also speculated that the decision to pursue VLIW was driven by the prejudices of a few researchers, rather than by sound technical analysis.”
The rest of this PDF is well worth reading:
http://courses.cs.washington.edu/courses/csep590/06au/projec...
And I don't actually disagree that VLIW is best suited for some specific workloads as opposed to general workloads. All I'm saying is that those workloads, such as decoding, video and a few others, are becoming more important as the consumer smartphone workload and that most smartphone CPUs can't do much more than 3 instructions in parallel anyways, which is close to the generally agreed 2-3 ilp that VLIW compiler can extract. Power consumption is heavily restricted on a smartphone as well. That being said, data movement is the big issue nowadays, so I'm not sure how much that will help.
I would be interested in learning more about the reasoning which caused AMD to move away from VLIW in their GPU designs since that seems like a relatively favorable area: very performance sensitive, people put time into optimization and tooling, and they don't need to run arbitrary legacy business apps.
By the ways, the NOP padding argument never convinced me because you can just use variable length instructions, and even current x86 decoders are variable length.
Also, many VLIW purists will claim (with some veracity also) that Itanium's EPIC != VLIW