Is that available to any core or just a sum-all-up and has latency penalties when going around?
depends on NUMA node config so I believe this is combined on the whole chip if all cores are working on threads with their local 16GB HBM, theoretically.
GPUs use 4-6 stacks of HBM2 which is 1,840-2,760 GB/s. It's 2x-3x the bandwidth of M1/M2 Ultra.
So 6,400 Gb/s compared to 409.6 Gb/s once you convert units?