HBM has such high access latency that the Xeon Max can't actually use all of the available bandwidth except in a caching mode (where a lot of that bandwidth is used with cache line spills and fills): the cores' line fill buffers will get full and cause backpressure and stalls in the CPU. And this assumes that every core is trying to load all the time.

You can see line fill buffer issues with 8-socket machines today, for example, where they are the cause of stalls due to the high latency of cache coherency checks. However, since they are part of the core and fine for 99% of users, the size of the line fill buffer is kept relatively small.

That limitation puts the HBM Xeon Max sort of on par with Grace for actual usable memory bandwidth, not so far above it.