Comment by supernova87a | Hacker News Reader

Comment by supernova87a | Hacker News Reader

This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong.

Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more.

He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter.

Of course I'm sure an expert in the field could tell instantly.

I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome?

He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that.

I would love to know what industry people here think, and then I might study the video / findings with more attention.

A curious aspect of this is that Intel has been doing very well financially despite having been failing technologically for a long time (if you buy the old story that Intel's process lead was at the center of it's market leadership.)

High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance.

---

Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc.

Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes.

Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles.

Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs.

Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.

I don’t understand your point that Nvidia (or ATI) would have gone under if it weren’t for deep learning or crypto.

Crypto on GPU was only a relatively short lived flash in the pan, and deep learning has only been a meaningful revenue generator in the past 5 years or so.

GPUs were doing just fine without that.

Demand for crypto bent the demand curve at a time when GPU manufacturers needed cash and an optimistic outlook to tell investors.

The astonishing thing was the long life of the 10-series; if the 10-series was an airplane it would be the 737. Hardware innovation has usually gone faster with GPUs, but the long life of the 10-series realized that performance because software had a chance to catch up.

There’s a steady ~2 year cadence at which new GPU families are introduced. The 10 series was nothing out of the ordinary.

Kepler - Maxwell: 28 months

Maxwell - Pascal: 20 months

Pascal - Turing: 25 months

Turing - Ampere: ?, but rumors have if for sometime September, so ~25 months

And Nvidia has never been in need of cash.

The crypto boom was just a nice extra boost, followed by a nasty bust.

One could very argue that the crypto boom was a net negative.

> A curious aspect of this is that Intel has been doing very well financially despite having been failing technologically for a long time.

So did IBM, for a long time. Current revenues are the product of past effort.

This is the same guy who loudly proclaims Apple moved away from Intel because of bugs in Skylake and subsequent (x)lake.

That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts.

At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver?

It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC.

But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.

Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish.

Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great).

The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.

I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ).

AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints).

You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).

Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.

RISC-V is also adding Vector extensions. The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one.

https://gms.tf/riscv-vector.html

I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.

The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.

> so code can take advantage of newer hardware / degrade gracefully on less capable one

I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff.

1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such.

2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.

Doesn't AVX-512 have a problem with frequency scaling?

https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...

https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html

Given this, and the fact Ryzen doesn't support AVX-512, I would be very wary of the technology.

From the comment you are replying to:

> The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.

I've worked on a project to compile a domain-specific description language down to both SIMD (by using Intel SPMD Program Compiler) and GPGPU (by using CUDA and OpenCL) Monte Carlo simulation code. We also have a Scala/Java interpreter for debugging.

Are there workloads that benefit from very wide SIMD vectors that aren't good fits for GPGPUs, as long as the GPUs support 64-bit floats close enough to IEEE-754 for your needs? I understand the overhead in shuffling data between main memory and GPU memory, and synchronization overhead, but most code I'm familiar with that does heavy number crunching suitable for very wide SIMD tends to do that number crunching off on threads that don't have much synchronization with threads doing more general-purpose computation.

On a side note, ISPC's input language is deceptively close to C, but little traps lie in wait. I remember helping an intern debug his port of some of the Java code I wrote over to ISPC, but it turns out that mulitplying a long long by a double in ISPC results in a long long. Our attempt to scale down a random 64-bit integer to the range [0.0, 1.0) was always resulting in 0.0. I get that integer calculations are faster, and I could understand disallowing implicit casts, but making a language so close to C, but with different implicit casting rules is just asking for trouble.

You're in a bit of a different domain - I've never really done all that much stuff with heavy number crunching (I've done a bit of work on random forest traversal, but that's more about logic than about FP). And I have never worked with ISPC.

I think number crunching workloads are typically quite suitable for GPGPU - I'm certainly not trying to "debunk GPGPU", just saying that there are a lot of integer/logic intensive workloads that involve rapid switching back and forth between control/GPR-based-logic sides and "SIMD tasks" (e.g. Hyperscan switching between NFA/DFA simulation and "acceleration", which was SIMD-based character skipping).

Dan Luu does brilliant things parsing strings with SIMD instructions but the programming model is busted if you have to recompile (and maybe hand code assembly) for each chip.

I was involved with wrapping up a deep neural net application just as GPGPU was about to take off. We used hand coded SIMD, performance was fine, but we were not going to recode for the next generation from chips, change our error function, or change anything SIMD related.I

Before there was MMX computers came with vector units that did not have a programmer visible size, such as the Cray 1 and the vector unit for the 3090 mainframe. As it is, people hardly use vector instructions or if they use them it is something a few generations old so people pay for vector units that they don't use.

simd is a better fit than gpu for a lot of data warehousing workloads where you aren't doing much computation on each individual value and it's not worth the overhead of sending the data to the gpu. also true for search index performance optimization.

I'm sold, and not sold. Having parsed terabytes of JSON, I was impressed by your work. Regexps and other operations are common-place, and if libraries like yours were in broad use, I could definitely see the appeal.

I'm sold on this as a technical pathway to performance.

Problem is, I do it in Python. I have a hard time laying out the breadcrumbs between simdjson and hyperscan, and practical, general-purpose code running more quickly on Intel CPUs. My code will run faster on AMD CPUs in practice even if, in abstract, given a few extra hours of coding, it could run faster on AVX-512. I'd rather wait a few hours for my computer than port my code to simdjson. I think that's true for 99% of the code out there.

Didn't we go down this path with Pentium IV's deep out-of-order pipeline and Itanium's VLIW before? Where Intel had great theoretical performance, but software didn't keep up?

For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions. That's perhaps more practical than in the Itanium and PIV era, but that doesn't mean it will happen.

> Where Intel had great theoretical performance, but software didn't keep up?

> For this to make a dent, Intel would need to hire a teach of engineers so Firefox and Chrome both took advantage of the new instructions.

That's more or less Intel's standard practice with its Intel compiler suite and MKL-and-similar libraries.

If there are performance gains from SIMD being left on the table, some of it is on Intel for not building the middle tooling necessary to make it easy.

(1) Most software isn't built with the Intel compiler suite or the MKL-and-similar-libraries.

(2) Even if Firefox/Chrome were built with those, they JIT Javascript. It's the JIT that matters, not what compiles the browser.

> AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten.

Speaking of Daniel Lemire, he recently published a paper https://arxiv.org/abs/1910.05109 mentioning downclocking in section 4:

> On some Intel processors, AVX-512 instructions are subject to downclocking: whenever AVX-512 instructions are executed, the processor lowers its frequency. We do not observe any downclocking on our processor, nor did we find any Intel documentation referencing downclocking for this processor. To test for downclocking, we used the avx-turbo benchmarking tool: it reports the processor frequency after issuing long sequences of expensive instructions of various types,including AVX-512 multiplication and floating-point instructions.

"...SIMD is underutilized in general purpose workloads... Daniel Lemire and I built simdjson to show the power of SIMD in a non-traditional domain..."

To your point, the recently front-paged simdjson writeup made me regret ignoring SIMD. I had dabbled in 3D graphics and matrix math stuff in the 90s and somehow continued to associate those extensions with that kind of work.

My bad.

Thank you for daylighting this stuff.

Since you sound quite expert -- I would love to learn from you (or others), at a beginner's level:

Is the design + production of a chip an exercise in optimizing the combination of:

-- size of die (and therefore overall cost/yield?)

-- power consumption

-- number / type / purpose of compute and memory subcomponents packed into the area

-- speed

Is that basically the envelope that you're doing an optimization within, and until a technology jump, even Intel/AMD/ARM-based are all simply choosing values between these dimensions? And each different choice of parameters within these basically goes after a certain segment of user?

Thanks!

I am not a hardware design expert, but... yes. Everyone makes their choice between speed, power consumption and cost. I attended a event once at Intel where someone told the anecdote that, "If someone comes to me and says, gee, I'd like a chip that runs faster, consumes less power and is cheaper to make I tell them 'you are overpaid for whatever it is that you do'".

The technology jump is of course the thing that allows you to "have it all" - until it doesn't. So it's clear that the optimization of Intel's 14nm++++++ is better for speed right now than anything they have at 10nm, which is better for density and power consumption. It used to be that the shrinks were unambiguously better but we've entered a strange new world.

Honestly, from the design perspective, I'm a enthusiastic spectator - to the extent I have insider knowledge it's out of date, poorly understood and not shareable in any case.

Microarchitecture is still very important - a better uarch and get you big effects without process changes. The last unambigiously successful one of these was Broadwell to Skylake. I think it will be fascinating to see the backport of Sunny Cove to 14nm ("Rocket Lake") as the difference between Rocket and Comet Lakes will give us a great natural experiment in how much microarchitecture changes matter without the usual distractions of process changes.

That used to be the "tick" in "tick/tock" but the uarch change is much bigger than usual and the process changes from Comet to Rocket will (I think) be quite small.

> partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off

So, Intel's problem is that it is now run by marketing droids, not engineers.

There are no good equity analysts. If they were good they’d run their own funds.

Reply on news.ycombinator.com