Agents went from 10% to 30% reliable this year, which is still a big deal.
lol
Yes! but, when they work, they only kinda work, sort of.
This is from a Dec 2024 which feels like a while ago
Hardly anything changed
AI is a going to be the next bubble. It can't even figure out who the real author of a sculpture is. It's really all BS made up to play with markets and geopolitics. Enjoy it while it lasts.
"We test baseline agents powered by both closed API-based and open-weights language models (LMs), and find that the most competitive agent can complete 30% of tasks autonomously."
So you ask it to try every task 3.33 times for guaranteed success?