Hacker News Reader

Digits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32)

Beyond that, the factors seem reasonable for 2 decades?

View on HN

Why even use a floating point if you have only 4 bits? Models with INT8 features are not unheard of.

1 sign and 3 exponent bits. AFAIK at the small number of bits it’s basically a teeny tiny look up table so you can precompute the table to be whatever math you want. Having an exponent instead of mantissa just means that the values that can be expressed are not linearly separated.

> Having an exponent instead of mantissa just means that the values that can be expressed are not linearly separated.

Being expressed as 1 10 100 1000 is not really different than being expressed as 1 2 3 4. There's still only four bits, i.e. 16 different possible values, no matter how we decide to express that in human-friendly terms.

> roadrunner is petaflops of FP32

Isn't it actually FP64?

So i can find sources that claim both ;) I wasn't sure what to believe, and didn't spend more than 5 minutes digging for the real results, so i went with the conservative one.

Reply on news.ycombinator.com