Hacker News Reader

Worth adding on that note:

From JAX to VLIW: Tracing a Computation Through the TPU Compiler Stack, https://patricktoulme.substack.com/p/from-jax-to-vliw-tracin...

Google’s Training Chips Revealed: TPUv2 and TPUv3, HotChips 2020, https://hc32.hotchips.org/assets/program/conference/day2/Hot...

Ten Lessons From Three Generations Shaped Google’s TPUv4i, ISCA 2021, https://gwern.net/doc/ai/scaling/hardware/2021-jouppi.pdf

Thanks, that JAX writeup was interesting.

Sure. I did mention DSPs. But how many people write code for DSPs?

x86-64 SSE and AVX are also SIMD

SIMD and VLIW are somewhat similar but very different in the end.

True.

The ISA in this Anthropic machine is actually both, VLIW and SIMD, and both are relevant to the problem.