I couldn't figure out what the actual measured memory bandwidth to the onboard HBM2e memory chiplets are- how does it compare to, say, Apple's M2 Ultra?
[0] https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...
[1] https://macperformanceguide.com/blog/2022/20220618_1934-Appl...
is a good punchline from the report. The Apple cores have that problem but a lot worse. They are slow. It goes back to this flawed idea in the community that you can "just" "add" "more memory," when parts like the H100 have their memory size matched to the architecture (physical and software) of the dozens of CPUs on them.
I'm not sure why the conception persists that a 60W laptop part would be comparable to a 300W server part of the same process generations, let alone this particular part.