Hacker News Reader

Comment by w0utert | Hacker News Reader

One thing I don't fully understand is why these accelerators all have relatively small memory sizes? Is it purely because of cost?

The most expensive Tesla accelerator I see in this list has 16 GB, which sounds like a lot, but it's 'only' twice as much as the consumer-level GPU I have in my PC at home, and it seems like a pretty paltry amount for large-scale simulations. I'm thinking about volumetric modeling applications, for example, where storing a dense voxel grid as a volume texture would easily run over 30 GB even for quite modest grid size/resolution (e.g. 2K x 2K x 2K grid). It's exactly these kinds of things where a GPU accelerator could shine, without the need to implement spare sampling solutions or accelerator data structures that would be complex and severely impact performance on GPU architectures.

View on HN

It's not just the cost, at least not just the cost of the RAM. GPU ram is designed to be wider than desktop memory, meaning that you need more pins on the die for the same amount of memory. The IO in top end accelerators uses all the real estate available for pins, and the dies are already at the reticle limit, so to add memory you have to either:

1) Use desktop memory that allows more BW/pin but is also much slower per pin

2) Develop completely new memory just for your application

3) Put some kind of buffering between the chip and the ram, greatly increasing latency.

None of these is currently that appealing for the manufacturers. HMC was originally going to do 3 by doing 2, but HMC appears stillborn, as all the momentum went to HBM instead, and HBM cannot increase the memory count per pin nearly as much as HMC could.

NVIDIA makes a chip called the P6000 that offers 24GB memory. It isn't high-throughput (HBM2) memory but regardless, the chip will benchmark relatively close to the Volta V100 and costs about half. Here's a deep learning benchmark for comparison: https://github.com/rejunity/cnn-benchmarks

> that would be complex and severely impact performance on GPU architectures

Agree about the complexity. However, adaptive grids can be a huge win in terms of performance. The GPU doesn’t need to simulate the whole 8G grid nodes, it only simulates much fewer nodes, only in the areas of large gradients / higher frequencies of the simulated fields.

Here’s an old SpaceX presentation about their GPU simulations of rockets: https://www.youtube.com/watch?v=vYA0f6R5KAI

That was very interesting and helpful, thanks!

If I understand it correctly, GPUs use specialized high-bandwidth memory that is more expensive instead of commodity memory. See https://en.wikipedia.org/wiki/GDDR5_SDRAM.

HBM2 to be specific: https://en.wikipedia.org/wiki/High_Bandwidth_Memory#HBM_2

Reply on news.ycombinator.com