RVA23 Ends Speculation's Monopoly in RISC-V CPUs

This article has a whole lot of "it's not X, it's Y"…

In reality this isn't much of a change. For decades it's been a given that mainstream CPUs have vector instructions. RISC-V was the odd man out in _not_ mandating vector instructions. Even so, most CPU code doesn't use them.

And this is unlikely to change anytime soon. Yes, ML workloads are becoming much more popular, but CPUs are still not parallel enough to do a good job at them. Only occasionally is it a good idea to try anyway.

Edit: Note that there is something novel about the approach that RISC-V and ARM are now following, namely being vector-length agnostic, but this is unlikely to have much impact on how much CPU code is vectorized in the first place. It improves scalability a little, but also gives compilers a little harder of a job. It is not something that's going to fundamentally transform the extent to which CPU code uses vector instructions.

The article makes some serious confusions, which make its conclusions wrong.

If the vector extension of RISC-V becomes mandatory, that has several good consequences, which are mentioned in the article. However, this has nothing to do with speculation.

There are many examples of CPUs which had excessive speculation or badly implemented speculation, for instance the infamous Intel Meltdown case, which was a mistake inexcusable for beginners, much less for experienced CPU designers.

Despite this, today it is impossible to make a CPU with acceptable performance without speculative execution, even if it should always be a design target that any failed speculation should not affect any kind of state that can affect the observable behavior of the CPU.

For example, not only any conditional branches require speculative execution, to avoid stalling the execution pipeline, but also a large part of the instructions may generate various kinds of exceptions, so they must be treated as conditional jumps that are predicted to be not taken, and only when an exceptional condition occurs, all the following instructions, which have been executed speculatively, must be canceled and the exception must be handled.

The requirement for speculative execution is caused by the existence of pipelined execution, which cannot be avoided for either scalar execution or vector execution.

Vector instructions do not help at all for avoiding speculative execution. The usefulness of vector instructions is that they ensure a much higher energy efficiency than in the case when out-of-order and/or superscalar execution is used to achieve the same performance, or than in the case when the number of cores is increased to achieve the same performance.

Whenever the solution of a problem is written by the compiler or by the programmer with the help of vector instructions or matrix instructions, that is a more efficient solution than an alternative that relies on more cores or on superscalar execution or on increasing the clock frequency, but this has nothing to do with speculation.

It looks like the author may work for Andes. If their out-of-order implementation is weak I could see why they would push vectors as an alternative.

This article is a complete waste of time. It reads like a children's story or a marketing announcement but it's not actually saying anything meaningful or making any technical point beyond just stating "If we use vectors then maybe we don't need speculation" but without providing much evidence except that highly parallel workloads already have enough work to do. Go figure. It mentions in the article we already have GPUs for this. CPUs are famously burdened with workloads that usually aren't GPU workloads. But now there's a declaration of a RVV requirement or something. (I say this as a SIMD programmer who likes a lot about RVV)