Hacker News Reader

Comment by fortran77 | Hacker News Reader

They're not really "grafting" it, they're implementing it as a first-class data type.

Definitely feels click-bait-y.

Ditto for them calling it "Brain Floating Point" [1]. I mean it appears to be a typical floating-point numeric data type with some of the precision truncated to reduce its cost.

I guess they might be trying to make it sound super-fancy to preempt people seeing it as cheap or/and low-tech?

[1]: https://en.wikipedia.org/wiki/Bfloat16_floating-point_format

It's called brain floating point because it was developed by Google's "Google Brain" machine learning project.

Yeah, I get the marketing perspective; just, we typically describe primitive data types in terms of what they are from a technical perspective rather than a marketing perspective.

fp16 & fp8 refer to IEEE 754 floating point which has notable differences from bfloat16 & make them have worse performance for machine learning.

Yeah the point is 'reduced precision FP16' would make a lot more sense - since it actually describes what it does. 'Brain' floating point makes you wonder how Neuralink are planning to use this datatype.

It's not really first class if you have to CVT BF16 to FP32 if you want to use FMA.

FMA on BF16 uses FP32 for the accumulator for accuracy reasons, but you can have instructions for "Rm = Rm + low(Rn) * low(Rp)" and "Rm = Rm + high(Rn) * high(Rp)", and they can be faster than FP32 "Rm=Rm+Rn*Rp" because there are fewer bits in the mantissas.

Also, converting BF16 to FP32 and back is just a vector shuffle that sticks/drops 16 extra mantissa bits at the end of each FP16 value, so it's cheaper than other floating point conversions. This means that even if you occasionally have to escape to FP32, the overhead is low and you keep all of the memory bandwidth benefits.

That's a good analysis. I particularly like the idea that if the overhead is low it gets hidden in the memory BW savings.

Reply on news.ycombinator.com