Hacker News Reader

Comment by dpe82 | Hacker News Reader

In a previous generation, the enabler of all our computer tech innovation was the incredible pace of compute growth due to Moore's Law, which was also "top-down" from very well-funded companies since designing and building cutting edge chips was (and still is) very, very expensive. The hype was insane, and decisions about what chip features to build were made largely on the basis of existing vendor relationships. Those companies benefited, but so did the rest of us. History rhymes.

View on HN

Should probably change this to "was appearance of incredible pace of compute growth due to Moore's Law," because even my basic CS classes from 15 years ago were teaching that it was drastically slowing down, and isn't really a "law" more than an observational trend that lasted a few decades. There are limits to how small you can make transistors and we're not too far from it, at least not what would continue to yield the results of that curve.

The corollary to Moores law, that computers get twice as fast every 18 months, died by 2010. People who didn't live through the 80s, 90s and early 00s, where you'd get a computer ten times as fast every 5 years, can't imagine what it was like back then.

Today the only way to scale compute is to throw more power at it or settle for the 5% per year real single core performance improvement.

We sort of had two phases of development:

- Running up single-core performance through increasingly sophisticated core design and clock speed (which is now at the 5% per year point mentioned)

- Going wider by throwing more SMT, more cores, and larger caches at the problem.

Assuming here that x86 was the last major architecture that was going for high single-thread performance at all costs, the first phase lasted us a good 30 years-- from the 4004 to the flameout of Netburst.

We could consider the second phase starting when they started delivering the P4 with Hyperthreading, and its true-dual-core predecessors shortly thereafter, so we're now about 20 years into that era.

Do we have another 10 left in it?

The difference is once you bought one of those chips you could do your own innovation on top of it (i.e., with software) without further interference from those well-funded companies. You can't do that with GPT et al. because of the subscription model.

Yes you can? Sure you can't run GPT5 locally, but get your hands on a proper GPU and you can run some still very sophisticated local inference.

You can do some, but many of them have license restrictions that prevent you from using them in certain ways. I can buy an Intel chip and deliberately use it to do things that hurt Intel's business (e.g., start a competing company). The big AI companies are trying very hard to make that kind of thing impossible by imposing constraints on the allowed uses of their models.

Eh, if this is true then IBM and Intel would still be the kings of the hill. Plenty of companies came from the bottom up out of nothing during the 90s and 2000s to build multi-billion dollar companies that are still dominate the market today. Many of those companies struggled for investment and grew over a long timeframe.

The argument is something like that is not really possible anymore given the absurd upfront investments we're seeing existing AI companies need in order to further their offerings.

Anthropic has existed for a grand total of 4 years.

But yes, there was a window of opportunity when it was possible to do cutting-edge work without billions of investment. That window of opportunity is now past, at least for LLMs. Many new technologies follow a similar pattern.

What about deepseek r1? That was earlier this year - how do you know that there won't be more "deepseek moments" in the coming years?

Intel was king of the hill until 2018.

“Bobby, some things are like a tire fire: trying to put it out only makes it worse. You just gotta grab a beer and let it burn.”

– Hank Rutherford Hill

You completly forgot about the invention of the home computer. If we would have all been loging into some mainframe computer using a home terminal your assessment would be correct.

Reply on news.ycombinator.com