Hacker News Reader

No more "stochastic parrots" and "LLM's can never produce anything novel, just regurgitate" comments anymore huh?

i still havent seen an llm invent new tokens/words over the course of a session?

the novelty remains a structural problem with how they work and what the weights try to predict

Sure, looks like 5% of math can be solved by 1T parameters stochastic parrot after NN trillion attempts (burned tokens). By numbers it could be less impressive than some brute force distributed chess engine.. Prove me wrong.

It's obviously significantly better than brute force. A trillion is absolutely minuscule compared to the size of the search space (like, 'rounding down to 1 is basically the same' small).

trillion indicates that its still brute force and not some search based on solid intelligent ranking function.

Also, I am not expert in this specific topic, but base on comments looks like claude parroted few recent works?

Reply on news.ycombinator.com