Hacker News Reader

There's actually an old paper titled Optimal Brain Damage, where they don't try to find optimal quantizations, but optimal sparse versions of a models-- i.e. where some weights are set to zero.

View on HN

I found…

Optimal Brain Damage https://www.researchgate.net/publication/221618539_Optimal_B...

Optimal Brain Compression https://openreview.net/pdf?id=ksVGCOlOEba#:~:text=The%20resu....

TinyVolt’s implementation of it: https://github.com/TinyVolt/optimal-brain-compression

Thanks, I'll read it up. Interesting.

That was really for them. You’re out there building neat stuff. Your talent might warrant looking into AdderNets and Bitnets which might get the cost down. There’s also some brain-inspired designs.

I don’t think many people have implemented such things. You might discover something new experimenting with them.

:)

Interesting, I heard something like that, but now I must read about it.

Reply on news.ycombinator.com