Even if you're all-in on OpenAI, does it not make financial sense to have _some_ stake in Nvidia considering they are the only ones with an actual moat?
Unless there are CUDA alternative breakthroughs we will hear about in the next few days.
https://finance.yahoo.com/news/amazon-says-anthropic-will-us...
> Anthropic is using 500,000 of Amazon’s Trainium2 chips for its Claude AI models as part of the tech giant's Project Rainier
They did not. They sold MOST of their stake in Nvidia long ago before anyone paid attention to Nvidia vs AI. It's one decision as part of a complex cloud of decisions.
Also this "stake" was around one thousandth of Nvidia. Minor. Most companies have lots of owners with much larger stakes.
AI inference on the other hand is basically just very large floating point tensor matrix multiplication. What does an ASIC for matmul look like? A GPU.
> What does an ASIC for matmul look like?
A systolic array, and ultimately quite different than a GPU. This is why TPUs et all are a thing.
In general with a systolic array you get a quadratic speedup. For example with a 256x256 array, it takes 256 cycles to shift operations in and out, but in doing that you accomplish 65k MACs in 512 cycles for a speedup of 128x over serial.
In the big picture, TPUs are systolic arrays. They don't have threading, divergence, or similar.
GPUs in the big picture are SIMT, a hybrid of SIMD and multithreading where individual data streams in SIMD are relaxed to allow them to diverge somewhat.
Memory Wise the TPU can keep the partial products right in the array. Parameters and weights are held in large on die scratch memories, and backing those are streams coming from the HBM. The TPU acts as a single giant CISC coprocessor, and has much more predictable memory and communication patterns vs a GPU, which its design exploits for higher efficiency at inference vs GPUs.
So even if they use the word "tensor" and both have HBM based memory systems, how those are actually architected is very different.