Hacker News Reader

Comment by freediver | Hacker News Reader

I’ll dare make a prediction: Transformers will be declared a dead end in AI in few years and GPT-4 might be the last in line. The method is able to produce highly convincing gimmicks (which is what it comes to once initial excitement splashes) but that is not how art or intelligence works.

View on HN

> The method is able to produce highly convincing gimmicks (which is what it comes to once initial excitement splashes) but that is not how art or intelligence works.

Uh, art is often about reproducing highly convincing gimmicks.

See https://en.wikipedia.org/wiki/Realism_(arts)

I think you mean that Transformers will become kitsch, which probably.

We've already seen this to some extent with GANs - which now look somewhat outdated in comparison with Dall-e and others

Language model != Transformers. The scaling power of the transformer architecture allows LLMs to exist, but LMs are just one application of transformers. The architecture itself is extremely generic and is applicable to nearly every problem.

I'm pretty certain whatever comes after GPT-4 will still be using layered self-attention.

Everything is eventually called a dead end in AI, no? Once it has moved from the bleeding edge to something the public uses, the limitations become known and everyone says, "of course a computer can do it, but that's not intelligence."

That's because we aren't building systems capable of self-analysis and awareness. As far as I'm aware, there are only AI/ML systems capable of analyzing some data and giving some output with little to no nuance

no, the OP says it in a different sense, namely that even if you are duly impressed by these advancements, they will stop in a few years and a new line of research will be needed

Reply on news.ycombinator.com