Hacker News Reader

Comment by rock_artist | Hacker News Reader

I'm not in this field as others. so I'd really appreciate a comment from some experienced HN folks.

I wonder what's the end goal.

The ML/AI world seems to be changing fast. So one model approach might become "old tech" if something better comes.

Is our current state of the models is stable so the LLM glory would be the same in 1-2 years? Or suddenly there'll be new approach making this "tech" obsolete?

View on HN

Historically speaking there is a good chance the transformers approach will be replaced by a new approach or a substantially altered transformers approach in the next five years. Efficiency will increase both on the software and the hardware front simultaneously, there is a lot to be optimized with the same parameter count.

What looks state of the art will probably look to people in 20 years how the 1885 Mercedes Benz car looks like to us in terms of efficiency, right now it's a bit of a bruteforce approach in general

Perhaps some sort of memristor crossbar array that could process large array calculations efficiently? Something like that could be pretty universal, until the tech moves to spiking neural networks. And even then the knowledge you gain from advancing on that path would likely transfer over.

there are already dozens of use cases that would become (more) profitable/interesting if we had chips optimized for LLMs

they were asic for BTC at a way earlier stage

+they can always pivot in a year if the market changes too much

> they can always pivot in a year

I think you are downplaying a bit the timescale of going from ideation to tape-out in ASIC design.

Reply on news.ycombinator.com