227 karma · joined January 22, 2017
Intel (for better or worse) takes a very experiment-results-driven approach to choosing which features to implement in hardware. So this result -- that software emulation of a feature works almost as well as a hardware implementation would -- probably makes Intel less likely to implement the feature in hardware.
ARM, AMD, RISCV etc, will probably come to similar conclusions.
It's possible that some other custom format is better in absolute terms for models trained specifically for the custom format. But for the current ecosystem, where models are trained primarily using float32, bfloat16 is a very good choice.
https://www.haiku-os.org/legacy-docs/benewsletter/index.html
Pachter Factor on SIFTD Games channel https://www.youtube.com/channel/UCFaBUWXO8o4jSfF6PhJOyZA Discusses the game business.