Hacker News Reader

I wonder what inference of big LLMs might look like with that much cache and memory bandwidth. Not trivial to get a benchmark for that but I wonder

For LLMs you'd want to use the MI300X variant and benchmarks should already be available.