Hacker News Reader

Comment by Jimmc414 | Hacker News Reader

This could be a very big paper if its claims are reproducible. Like approaching attention is all you need big.

They discovered 106 new state-of-the-art linear attention architectures through a fully autonomous AI research loop. The authors are making comparisons to AlphaGo’s move 37.

View on HN

>This could be a very big paper if its claims are reproducible. Like approaching attention is all you need big.

If it were it would have the on of the worst title imaginable and include one of the worst abstracts for any such paper.

It seriously is a very big red flag that a paper is doing this much to talk about how important it is. It is also essentially pre-writing the headlines for journalists to use, making this whole thing, at least, look like it is a PR stunt.

The part that is in principle amenable to replication is where they throw a lot of stuff at the wall and see what sticks. The part where they hype their own work, on the other hand... as a rule of thumb, if this really were a breakthrough on the level of AlphaGo, they wouldn't have to make that comparison themselves, someone else would be impressed enough to do it for them.

Let's definitely wait for replication, but I am honestly not that surprised that it works. I am surprised it took so long for people to give it a real try. It's such an ideal scenario: every experiment is conducted inside the computer, so there is no need to gather data in the real world, which is the pain point for most experiments in science. The LLM is therefore free to try a lot of different combinations and learn in real time what works and what doesn't.

Reply on news.ycombinator.com