Hacker News Reader

Nvidia has insane margins so it isn't hard* for a well-funded player to beat them in FLOPS/$. The hard part is the compiler toolchains, the libraries, the documentation, getting models to run fast.

* Everything is relative. It's hard to make even a simple microcontroller - but also it isn't hard, you know?

"Life in 1 comment"

> Nvidia has insane margins so it isn't hard* for a well-funded player to beat them in FLOPS/$

It's hard when you can't buy TSMC capacity

I’d wait till the dust settles. Maybe 2bit encoding (-1, 0, 0.5, 1) would be easier to design hardware for; as 0.5 can be a bitwise shift.

Nobody cares about FLOPs anymore when it comes to transformer based architectures and considering the transformer architecture is applicable to almost any usecase beyond LLMs, memory bandwidth and memory capacity are the most important metrics. Grayskull has enough 16bit float performance to outrun its memory bandwidth even with 1 ternary bit quantization.