Hacker News Reader

Before AI it was HPC that was obviously becoming huge. AI is the most recent and buzziest form of HPC, but people have been buying AI-style clusters for a while. The Knights [*] products were a great entry point into that market if they would just figure out the software of it. Knights [*] essentially had Atom cores with huge SIMD units, and fast forward to today, an AMD GPU is essentially a small scalar core attached to a huge SIMD unit.

IMO it seems like they had the Google problem: if you can't sell a billion units it has to be canceled.

That's because Larrabee wasn't totally canceled, it was just pivoted from a gaming GPU to an HPC accelerator in the form of Knights Corner/Landing. The idea of having lots of x86 cores with wide SIMD units didn't change, but it was a lot more successful in the HPC world because anybody could compile an old MPI or OpenMP application with AVX512 and it just worked.

To some degree, but the first cards didn’t really give great performance in practice. By the third gen they were worth it and then Intel cancelled it.

Larrabee itself was a complete disaster. Xeon Phi resurrected the Larrabee trash heap into something sellable. Sadly, it was never really that easy to program a Xeon Phi.

> The big win no one could’ve predicted is that if you had a good modern competitive GPU five years ago, then you were well positioned to pivot some for AI

Nvidia pivoted to AI almost 2 decades ago (late 2000's). And big tech recognized GPU's potential around the same time.

Google brain was 2011 and tensorflow was publicly released in 2015 (much earlier internally).

The "Attention is all you need" paper came out in 2017. That was for many folks, another smoking gun that GPU's are fundamental to AI.

The point is: Intel has had a looong time to get into GPU's. They just chose not too. And it wasn't because "nobody saw ai coming".

The people intel hired to start a discrete GPU business were just perennial incompetents and played corporate politics instead of delivering.

In the old days gamers paid extra for sound cards etc., all these multimedia addons gradually got commoditised. I don't think it would've been a completely crazy bet to assume the same was going to happen to the GPU.

There were even rumors about "IGP is all you need" and Intel "canceling" Nvidia by eliminating PCIe so there would be nowhere to attach a discrete GPU. (Intel did cancel nForce.) In retrospect that was completely ridiculous.

The big problem for them is AMD. Anytime they do something stupid (Itanium) they have a competitor right there to take advantage of it.

Same problem IBM had. They didn’t like people using “their“ PC architecture, so they decided to make the MCA bus and other proprietary stuff. Force everyone to do what they want.

IBM‘s AMD was Compaq and all the clone makers. They all got together and used their power to prevent the market from being screwed up by IBM.

I could really see Intel wanting to make that kind of move but it never would’ve worked in reality.

Itanium was not stupid. Some genuine effort behind it could have changed this whole story.

I'm curious why you think there was not effort behind ia64.

I still think ia64 was sunk by the let-the-compiler-do-it tarpit. There were ambivalent aspects such as, at the time, questions of power or chip area. But things like memory latency are just not predictable enough to do strictly in-order pipelines. OoO won.

Not the OP, but share a similar point of view.

Had AMD not been around with AMD64, the industry would have had to get it working no matter what, HP and Microsoft were already on the Itanium train, and if production of x86 got slowly replaced by Itanium that was it, Windows and HP-UX would drag the ecosystems into it, and eventually the remaining issues would be sorted out.

Wouldn't the world have jumped to Alpha instead? If you're going to switch the entire architecture, might as well switch to one which is sane.

Windows on Alpha didn't went far, so I guess no.

UNIX systems, I also doubt, as we were still in the days each vendor had their own CPUs.

No. HP was behind Itanium. In fact, it was basically their idea in the first place. I'm still not entirely sure why Intel went along although I suspect at least some of it was getting away from various x86 cross-licensing agreements.

Intel bought Alpha and killed it to eliminate the competition.

HP bought DEC by way of Compaq. Some DEC Alpha assets ended up with Intel (e.g. Hudson MA). But your statement doesn't make sense in general.

My issue with this alternative history is Intel’s consumer roadmap for Itanium, or rather its complete lack of one.

AFAICT, they expected Itanium to own the high end server/HPC market, while 32-bit x86 would continue supporting PC’s for another decade.

That seems remarkably short-sighted in retrospect.

I'm inclined to agree. Intel would have been loathe to go the 64-bit x86 route on their own. But a somewhat resurgent AMD pretty much forced their hand given the sorry state of Itanium development. Absent AMD, I'm pretty sure they'd have found a path to a "good enough" Itanium to sell successfully.

For what it's worth, I followed Itanium as an IT industry analyst and wrote a short book recently on the topic. https://s3.amazonaws.com/bitmasons.com/docs/Lessons+from+the...

Right. Even if Itanium worked better and was released to consumers AMD might have done the same thing and still won.

The compilers were fine, even if not producing maximally optimized code. Intel’s decision to add x86 compatibility circuitry to the die was probably the fatal one, it slowed everything down; made for terrible comparisons with existing x86 performance and generally signalled a lack of confidence. Something like Rosetta was out of the question, but they could have just had better transitioning tools for those code bases that couldn’t be recompiled easily.

Maybe early (say 94-99?). To some degree that happened. Most computers using an integrated GPU at this point, it’s mostly gamers or people who really need the power that have one on an expansion card.

Of course it’s not actually the CPU doing the work, it’s just in the same package.

Part of the difference is that audio can only get so good. But graphics don’t have much of a limit yet. We kept adding more colors and higher resolutions and higher refresh rates, more polygons, and even a second display for true 3-D.

It’s the true embarrassingly parallel problem we’re throwing more silicon at it just keeps making it better.

By the time Intel decided to get “serious“ and develop Larrabee, they clearly saw the market was there and would keep growing otherwise they wouldn’t have even bothered.

And honestly, whether it was on the same package as the CPU or an actual part of it the way normal vector units or the branch predictor are, who would benefit more from having a good graphics story to go on their GPU than the king of CPUs. They could use their advantages to just further cement their moat.

IMO the main reason why dedicated audio cards aren't a thing anymore is that Microsoft killed hardware accelerated audio as part of their driver API changes in Vista. The result is that games today have worse spatial audio than games that came out 20 years ago. If Microsoft hadn't done that, I think audio cards would have been in the same position as video cards where most users have their needs met by integrated solutions and gamers/high end users get dedicated cards (of course dedicated audio cards would have been more of a niche than GPUs).

Yeah there were old dedicated sound cards with support for spatial audio, and they were killed. But CPU's also just got really really fast in the meantime. You don't need dedicated hardware for spatial audio processing, it's cheaper and easier to do in software.

There have been enough attempts post-Vista to do spatial audio on GPUs that I don't think the argument that CPUs were fast enough really holds water. I think it's mainly that spatial audio hasn't been a must-have feature, so omitting it or half-assing it hasn't been a problem.

> The result is that games today have worse spatial audio than games that came out 20 years ago

This was already solved 7 years ago, but few games actually make use of it: https://valvesoftware.github.io/steam-audio/

I wouldn't call a private implementation of spatial sound "solving the problem". If tech were that simple we'd have solve a lot of non-trivial problems.

If you were to design a GPU several years ago, people would probably tell you designing it for AI is crazy and cryptocurrencies and raw hashing power is where the money's at.

In the end, Nvidia kind of got lucky with their bet on AI. While CUDA has been a thing for a while, it was limited to mostly academic and workstation workloads. It wasn't until LLMs and things like Stable Diffusion when it really became popular.

Unfortunately for non-NV users, most of these AI workloads when run locally assume CUDA is present and it's a considerable effort to make them work on non-NV GPUs. It's fairly okay for common workloads like running basic LLMs and SD models, but as soon as you get into more advanced/obscure stuff the harder it gets without CUDA.

> Nvidia kind of got lucky with their bet on AI

I see flavors of this narrative quite often. Nvidia has been positioning their GPU's for AI since the late 2000's. And their GPU's were being used for AI at least as early as 2011

To me, it doesn't seem fair to discount Nvidia's foresight. A lot of smart people saw this opportunity even before the invention of LLM's. That's a bit like building a light bulb before discovering electricity.

I do think it's unfair to discount Nvidia's success as "having gotten lucky." On the other hand, I also think the back-to-back 1-2 punch of crypto and LLMs was probably hard to foresee as something way more important than a handful of high-end gamers and maybe some commercial linear algebra applications.

> You certainly couldn’t know AI was coming.

?? People have been buying GPUs since the late 90's, it was always an obvious growth vector.

And forget AI, we were using GPUs to run statistical programs since the late 00's...

I remember having this argument 10+ years ago. Discrete GPUs like what gamers were buying were thought to be a small market with no real routes to bigger markets. Workstation and HPC could be addressed with entirely separate product lines, hence the disaster that was larrabee. It was Nvidia that pushed the unified workstation/consumer architecture. ATI reused core architecture, but they had very different software stacks above with their firestream stuff.

> the nonsense of Larrabee/making it out of x86

It only seems nonsense in retrospect. No one expected nVidia to improve flops/dollar in their GPUs that quickly.

And Larrabee wasn’t the only architecture deprecated by GPGPUs, Cell Broadband Engine by Sony+Toshiba+IBM was another one.

Cell required Assembly programming Demoscene skill level to make good use of them, there was a C sdk, but really you wanted to write Assembly, and relatively tricky Assembly at it.

It is one of the reasons why Playstation 3 is the less celebrated one, and the console generation where XBox was able to pull ahead.

Larrabee, I was at a GDCE 2009 session where it was demoed, and while the idea was promising, it didn't seem to actually pull off in practice, plus there was hardly much hardware where we could actually try it out ourselves.

Xeon Phi didn't made it better regarding adoption, nor did AVX.

> Cell required Assembly programming Demoscene skill level to make good use of them, there was a C sdk, but really you wanted to write Assembly, and relatively tricky Assembly at it.

> It is one of the reasons why Playstation 3 is the less celebrated one, and the console generation where XBox was able to pull ahead.

In retrospect, Cell was a great architecture. In typical Sony fashion it was simply a few years too early. At the time games were single-threaded and many console games still expected CPU and GPU to be in sync.

On the PS2, the difference between using the vector accelerator or not was 1.4x – many developers never bothered. But on the PS3 this was simply not an option, as the performance difference was 6x (!).

The big issue at the time was dev tooling. Most existing engines had no support for dispatching jobs or any form of pipelining. Adding that required redesigning the engine from the ground up. On top of that the dispatcher/manager and the jobs use two different µarchs. Obviously, game and engine developers at the time hated cell with a passion.[1]

But even on the PC and Xbox, frequency scaling ended with the Pentium 4. Multi-core CPUs became mainstream. As CPUs stalled, most of the performance increase on PCs came from GPUs, that had moved from a fixed-function pipeline to shaders.

Slowly, even PC games had to do the bulk of their work in jobs dispatched across multiple cores. Engines that had adapted to Cell got a massive head start.

Today, the very textbook for game development – "Game Engine Architecture", written by Jason Gregory – uses the architecture of the Uncharted games on PS3 as the prime example for how to build a modern engine. As result, nowadays even small indie games are built that way.[2]

________________________

1. Gabe Newell: "Investing in the Cell, investing in the SPE gives you no long-term benefits. There's nothing there that you're going to apply to anything else" https://www.wired.com/2007/10/valves-gabe-new/

2. e.g., Tiny Glade: https://www.youtube.com/watch?v=jusWW2pPnA0.

Yes, in that regard it was ahead of its time, and nowadays many game developers are exploring the possiblities to get rid of everything and explore pure compute for a full software rendering stack, running on GPGPU, agnostic from the actual rendering API.

Although I think it would still be better to use specific shading languages as the hardware isn't the same architecture, thus there is only so much one can do with traditional languages without extensions, or GPGPU specific algorithms/data structures, but that is me as outsider with interest in the field.

The experts seem to be enjoying exploring CUDA, SYCL, MSL, and similar for such purposes.

That sounds fascinating… any pointers here?

For example OTOY uses CUDA renderering.

https://home.otoy.com/render/octane-render/

If you prefer actual code,

https://advances.realtimerendering.com/s2015/aaltonenhaar_si...

and

"Mesh Shaders - The Future of Rendering"

https://www.youtube.com/watch?v=3EMdMD1PsgY

"GPU driven Rendering with Mesh Shaders in Alan Wake 2"

https://www.youtube.com/watch?v=EtX7WnFhxtQ

About rendering triangle meshes with compute shaders, here’s a long technical video how UE5 does that: https://www.youtube.com/watch?v=TMorJX3Nj6U&t=3488s

> required Assembly programming Demoscene skill level to make good use of them

Yeah, but writing efficient CUDA kernels, or D3D compute shaders, is not particularly straightforward either. At least GPU cores have direct access to global memory but still, for good throughput that memory access needs to be coalesced. Then there’re manually managed groupshared memory, atomics, wave/warp intrinsics in D3D12/CUDA, and now these special low-precision matrix multiplication instructions for AI.

People accepted all that complexity because performance is too good to ignore, both flops/dollar and flops/watt. Contemporary mainstream GPUs were just better than SPEs on Cell, or AVX-512 on Xeon Phis.

Specifically, PS3 (2006) delivered up to 200 GFlops FP32, first-generation Xeon Phi (incredibly expensive devices released in 2010) 750 GFlops, but GeForce 460 (released in 2010 for the price of $200) delivered up to 900 GFlops.

True, but it is much easier than Demoscene Assembly skill level.

Same for AVX instrinsics. Very fun to use (for some of us), powerful when correctly used, but so hard to actually get right and to get the expected performance. I wish Intel had put more into ISPC back then.

Larrabee (or something similar) could have been an amazing real-time operating system target. Each little (Pentium!) core having truly its own local memory, no contention. But it was pitched as a GPU and the raw performance was not there.

Intel sucks at software and should have leaned in to their strengths - documenting the hell out of the low level workings of Larrabee. Workloads tuned to it would have been a great moat.

Another way to see this is NVIDIA invested heavily, heavily in making a language and its compiler useable (for some level of useable.......) on enthusiast-affordable and poor-scrappy-researcher-affordable hardware on a complex weird-to-program-from-asm architecture and provided good-enough performance (and showcasing lots of very good optimizations) on standard HPC libraries and a bit of HW/SW co-design and kept at it for a long long time... with some level of backwards compatibility too. The recipe is here.

Intel did a good job on iGPUs, which satisfy most customers (even back in the i915 days).

But in retrospect, of course it was a missed market. It's hard to disentagle that choice from others in the same era (including the refusal to jump onto EUV or chiplets).

They did a good job in the sense that they provide the minimum neccessary baseline to run a graphical Windows environment without additional hardware. The attitude that it's a good job because it satisfied most of their current customers is exactly the problem with the whole companies approach to product development.

A lot of the technology behind EUV is just insane, but it's not like ASML has done so great financially. CMOS process shrinks have been a pretty impressive unicorn. There are various levers but not clear to me what will happen going forward.