Hacker News Reader

Perhaps an analogy to Moore's law?

Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner.

What countered it: Moore's law stopped working.

Bitter lesson #2 similarly relies on scaling laws that might have diminishing returns wrt model runtime vs intelligence. Runtime matters for turnaround on the problem you're solving.

View on HN

Moore's Law has nothing to do with processors getting faster. Dennard scaling stopped working but Moore just slowed somewhat, not stopped.

You are technically correct. The best kind of correct.

However, what most people think of as Moore's Law--CPU speed doubles every 18 months--broke somewhere between 90nm and 22nm.

And even the actual Moore's Law--2x the transistors every 18 months--doesn't hold for all types of chips anymore. Memory only gained 2x density over 10 years.

Reply on news.ycombinator.com