Hacker News Reader

Comment by thomasahle | Hacker News Reader

288MB Cache. I wonder if we'll ever be given some control about how and what computations are cached. It seems like a lot of memory to leave to simple heuristics.

View on HN

Such control has been available for many years. On x86 you have "non-temporal" load and store instructions [1] and the Cache Disable bit in Control Register 0 [2] which can be used to suggest which bytes are not suitable for caching. C also has posix_madvise() [3] which is somewhat relevant.

[1] https://stackoverflow.com/questions/37070/what-is-the-meanin...

[2] https://en.wikipedia.org/wiki/Control_register

[3] http://man7.org/linux/man-pages/man3/posix_madvise.3.html

It's 32MB L3 per die and 512KB L2 per core, they sum them for marketing effect. Effectively one core can access "only" 32.5MB.

Errata: as on one die there are two CCXs, one core can access 16.5MB

I feel the cache in these modern CPUs is being used extremely efficiently. If they have a branch predictor that is 99% accurate, I have faith in letting the same engineers manage my cache eviction strategy. The extreme scope of modern OoO execution strategies would probably make detailed CPU cache management more of a liability than an asset.

One simple policy would be to try to ensure your application is as small as reasonably possible. If you can fit your entire executable image in a fraction of your L3 you are probably sitting in a really good spot.

I’ve heard these “simple heuristics” are multi-megabyte neural networks these days. I believe I heard that claim in this video: https://youtu.be/ymcOLL2qEg8

Usually branch prediction is what ANNs are used for in CPUs. Caches usually operate as complicated pseudo-LRUs.

https://stackoverflow.com/questions/22597324/what-cache-inva...

It's possible that Tesla does something different that makes sense in their specific situation, but general purpose CPUs - as far as I know - don't use NNs for this.

Ryzen and some ARMs use perceptrons for branch prediction. It's essentially a single-layer neural network with no nonlinearity at the end.

Multi-megabyte NNs for a normal branch predictor would be impossible; way too high latency to be useful.

> It's essentially a single-layer neural network with no nonlinearity at the end.

That's just a fancy way of saying matrix multiplication :)

No, matrix multiplication is just an implementation detail.

Didn't Intel try precisely that with Itanium, only for it to fail spectacularly?

Wasn’t itanium mostly the reverse, expectations that compiler heuristics would take care of the architectural complexity?

Reply on news.ycombinator.com