What's wrong with Intel, and how to fix it: Former principal engineer unloads

Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great).

The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.

I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ).

AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints).

You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).

High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance.

---

Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc.

Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes.

Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles.

Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs.

Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.

That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts.

At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver?

It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC.

But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.

I recall him mentioning exactly this.

And that's what is scary. Intel is failing left and right but the rap track from Intel sounds like an ad for Disneyland.

The problem with this type of system is both the software and OS have to be aware of what is or isn't a "main thread" job. You have to architect your whole program around job systems to spread your data between the main thread and sub-processes which was a major complaint of gamedevs for the PS3. There's also the fact that many consumer software developers just don't care and will flag all of their processes as high priority jobs.

It worked out decently well on PS3 because it was such a locked down system, but for a general purpose desktop it would be utter chaos.

I can't wait for real-world usage benchmarks of their ARM CPUs once actual production machines are out in the hands of actual customers. It has the potential to upend the entire laptop market.

In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts).

It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability.

The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512.

So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.

These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.

Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core.

Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).

He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly.

Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.

He also sold all the stock he conceivably could during his tenure: https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-sh...

Intel's founder, Andy Grove, used to say: "Only the paranoid survive". (He personally survived under a false identity in Nazi-occupied Hungary.) I'm afraid Intel had stopped being paranoid enough before it started to stumble more and more seriously.

Intel is a hardware tech business, its a complicated space that needs deep knowledge to understand it before you shape the future of it.

Technical people might not be the only ones that should run business but they should be the core of it.

I have same view as the video, engineers need more power and decision making not less to win. Intel will not fight back by cutting down cost of production by 5%. But by creating next gen chip.

But that all goes to pot when the technology is changing so fast, the battle between companies becomes a battle about who advances technology fastest. In a battle fought over engineering, the engineers have a home ground advantage.

It's impossible not to notice that Apple, Google, Microsoft and Facebook were all founded by true nerds and when you here their leaders speak, most engineers feel right at home with their reasoning. When Microsoft changed their leadership to a non-nerd, it started to wither and die. Fortunately for them, they fixed that.

Looks like his video is him promoting himself to investors to try to get himself on the Intel board of directors.

For example, with GCC you can set the baseline CPU features. You can also use function multi-versioning to provide faster functions for CPUs with newer features.

https://lwn.net/Articles/691932/

Note that there are many new machines (Atoms and low-brand stuff like "Pentium" branded CPUs - even recent ones based on Comet Lake) which don't have anything beyond SSE4.x). IIRC they don't even have BMI/BMI2 which is a huge pain.

For HPC they will always compile to use the full set of instructions and see how it performs. For some software there will be explicit checks and loading the right compiled code(compression, video effects etc). On a desktop most things are compiled for the minimum processor though. The CPU might still be able to use its new features like wide registers of the instructions are in just the right order.

If you run something with instructions your processor doesn't have it will crash.

Now, if you want to be smarter there’s a number of things you can do, some of which have compiler or OS support. You could, in your program, manually detect these features (cpuid) and then use them if available. You could have entire functions that are specialized for one architecture and pick the right one at runtime; some compilers can do this for you “transparently” with custom attributes. If the dynamic linker supports it, you could have it pick the right version of the symbol for your processor and your program is otherwise none the wiser which one gets selected; it just has to have the appropriate versions of functions. And at the extreme end you can compile entirely separate versions of your program for different base processors; on Darwin you can have a universal binary recognized by the kernel itself as supposing x86_64 (basic 64-bit features) and x86_64h (assume Haswell and above) and the right one will get loaded based on your hardware.

This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. For a more extreme case look to older POWER processors which were 4-way threaded, presumably because they were even worse on single threaded code.