Hacker News Reader

It seems we could use a new kind of memory that streams the weight data in, like GDDR in reverse.

Optane? (Too soon?)

Optane was targeting the latency gap, while HBF targets bandwidth. Given how LLMs work, HBF is perfect for offloading.

interesting!

yes! I guess future hardware designs will have something like that!