Hacker News Reader

Great read. Have you compared performance with other Llama models (3, 3.2) or have you just done benchmarking with 3.1?

Is there some intuition as to why 3.1 might outperform 3.2 on MI300X?

In this one we were only using 3.1 405B FP8. We took one model to simplify the setup and were mostly looking at the memory saturation effect. So basically we compared inference metrics of the same model. I suppose comparing 3.1 and 3.2 will be difficult as they are different models entirely. But open to ideas

Reply on news.ycombinator.com