Study identifies weaknesses in how AI systems are evaluated
oii.ox.ac.uk
oii.ox.ac.uk
Product testing (with traditional A/B tests) are kind of the best bet since you can measure what you care about _directly_ and at scale.
I would say there is of course “benchmarketing” but generally people do sincerely want to make good benchmarks it’s just hard or impossible. For many of these problems we’re hitting capabilities where we don’t even have a decent paradigm to use,
Ultimately we are measuring extremely measurable things that have an objective ground truth. And yet:
- we completely fail at statistics (the MAJORITY of analysis is literally just "here's the delta in the mean of these two samples". If I ever do see people gesturing at actual proper analysis, if prompted they'll always admit "yeah, well, we do come up with a p-value or a confidence interval, but we're pretty sure the way we calculate it is bullshit")
- the benchmarks are almost never predictive of the performance of real world workloads anyway
- we can obviously always just experiment in prod but then the noise levels are so high that you can entirely miss million-dollar losses. And by the time you get prod data you've already invested at best several engineer-weeks of effort.
AND this is a field where the economic incentives for accurate predictions are enormous.
In AI, you are measuring weird and fuzzy stuff, and you kinda have an incentive to just measure some noise that looks good for your stock price anyway. AND then there's contamination.
Looking at it this way, it would be very surprising if the world of LLM benchmarks was anything but a complete and utter shitshow!
Sort of tangential, but as someone currently taking an intro statistics course and wondering why it's all not really clicking given how easy the material is, this for some reason makes me feel a lot better.
Pair that with skipping all the important problems (what is randomness, how do you formulate the right questions, how do you set up an experiment capable of collecting data which can actually answer those questions, etc), and it's a recipe for disaster.
It's just an exercise in box-ticking, and some students get lucky with an exceptional teacher, and others are independently able to develop the right instincts when they enter the class with the right background, but it's a disservice to almost everyone else.
“The customers want lower latency at 30% load for unique queries.”
“Err… we can scale up for more throughput!”
ಠ_ಠ
Whatever happened with Llama 4?
It's not like there's a shortage of skills in this area, it seems like our one specific industry just has a weird blindspot.
When I talk about this with other CS people in my own country (Spain) they tend to refer similar experiences.
This isn't just that nobody cares about the truth. People 100% care! If you actually degrade a performance metric as measured post-hoc in full prod, someone will 100% notice, and if you want to keep your feature un-rolled-back, you are probably gonna have to have a meeting with someone that has thousands of reports, and persuade them it's worth it to the business.
But you're always gonna have more luck if you can have that meeting _before_ you degrade it. But... it's usually pretty hard to figure out what the exact degradation is gonna be, because of the things in my previous comment...
Human raters are exploitable, and you never know whether the B has a genuine performance advantage over A, or just found a meat exploit by an accident.
It's what fucked OpenAI over with 4o, and fucked over many other labs in more subtle ways.
GPT-4o's endless sycophancy was great for retention, GPT-5's style of ending every response in a question is great for engagement.
Are those desirable traits though? Doubt it. They look like simple tricks and reek of reward hacking - and A/B testing rewards them indeed. Direct optimization is even worse. Combining the two is ruinous.
Mind, I'm not saying that those metrics are useless. Radioactive materials aren't useless. You just got to keep their unpleasant properties in mind at all times - or suffer the consequences.
The general populace doesn't care to question how benchmarks are formulated and what their known (and unknown) limitations are.
That being said, they are likely decent proxies. For example, I think the average user isn't going to observe a noticeable difference between Claude Sonnet and OpenAI Codex.
Without benchmarks, you're down to evaluating model performance based on vibes and vibes only, which plain sucks. With benchmarks, you have numbers that correlate to capabilities somewhat.
So, yes, we just aren't going to get anything that's radically better. Just more of the same, and some benchmarks that are less bad. Which is still good. But don't expect a Benchmark Revolution when everyone suddenly realizes just how Abjectly Terrible the current benchmarks are, and gets New Much Better Benchmarks to replace them with. The advances are going to be incremental, unimpressive, and meaningful only in aggregate.
It's like you wanted to say "tech companies are bad", and the rest is just window dressing.
Of course they tout benchmark numbers because let's be real, if they didn't tout benchmarks your not going to bother using it. For example if someone posts some random model on huggingface with no benchmarks you just won't proceed.
Humans have a really strong prior to not waste time. We always always evaluate things hierarchally. We always start with some prior and then whatever is easiest goes next even if its a shitty unreliable measure.
For example, for Gemini 3 everyone will start with a prior that it is going to be good. Then they will look at benchmarks, and only then will they move to harder evaluations on their own use cases.
Regardless though, I think the marketing could be more transparent
This finding really shocked me
The more generous take is that you can’t benchmarks advanced intelligence very well, whether LLM or person. We don’t have good procedures for assessing a person's fit-for-purpose e.g. for a job, certainly not standardized question sets. Why would we expect to be able to do this with AI?
I think both of these takes are present to some extent in reality.
We struggle a bit with processing and extracting this kind of insight in a privacy-friendly way, but there’s certainly a lot of data.
And how do you do an apples to apples evaluation of such squishy services?
Publish the debate as~is so that others vaguely familiar with the topic can also be in awe or disgusted.
We have many gradients of emotion. No need to try quantify them. Just repeat the exercise.
https://www.happiesthealth.com/articles/future-of-health/hum...
It's a shifting goalpost, but one of the things that struck me was how some questions could still be trivial for a fairly qualified human (a doctor in this case) but difficult for an AI model. Reasoning, visual or logic, is built on a set of assumptions that are better gained through IRL experience than crawling datasets and matching answers.
This leads me to believe that much of the future for training AI models will lie in exposing them to "meatspace" and annotating their inferences, much like how we train a child. This is a long, long process, and one that is already underway at scale. But it's what might give us emergent intelligences rather than just a basket of competing yet somehow-magic thesaurus.
For example a test of “multiply 1765x9392” would have some correlation with human intelligence but it wouldn’t make sense to apply it to computers.
There are LLMs, the engines that make these products run, and then the products themselves.
GPT anything should not be asked math problems. LLMs are language models, not math.
The line is going to get very blurry because ChatGPT, or Claude or Gemini, are not LLM’s. Their products driven by LLMs.
The question or requisite should not be can my LLM do math. It can I build a product that is LLM driven that can reason through math problems. Those are different things.
A coworker of mine told me that GPT’s LLM can use Excel files. No, it can’t. But the tools they plugged into it can.
It's a bit like saying that a human can't use Excel files, but when given a keyboard, mouse and monitor connected to a computer running Excel, it can. But then obviously the "Excel usage" competency is in the human; not in the tools, and a cat for example cannot use Excel proficiently however many training hours it gets and however good the keyboard is.
Taking it back to the LLMs, it is clear to me that some modern LLMs like the one running ChatGPT can be integrated with tools in a way that makes them somewhat proficient with Excel, while other simpler LLMs cannot, regardless of the tools.
And there's a 50/50 chance they'll use the right tool for the job. I tried the math question above multiple times on gpt5 and it gets it right about 50% of the time. If i ask to "try again" it usually gets it on the 2nd or 3rd try. Most times that it's wrong, it's not far off but it looks deceptively accurate at first glance.
We took objective computers, and made them generate subjective results. Isn’t this a problem that we already know there’s no solution to?
That grading subjectivity is just subjective itself.
I'm still using https://lmarena.ai/leaderboard. Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them.
And when we get a prompt working reliably on one model, we often have trouble porting it to another LLM - even straight "version upgrades" such as from GPT-4 to -5. Your prompt and your model become highly coupled quite easily.
I dunno what to do about it and am tending to just pick Gemini as a result.
Even professional human evaluators are quite vulnerable to sycophancy and overconfident-and-wrong answers. And LMArena evaluators aren't professionals.
A lot of the sycophancy mess that seeps from this generation of LLM stems from reckless tuning based on human feedback. Tuning for good LMArena performance has similar effects - and not at all by a coincidence.
Capturing LLM performance with a single metric is a hopeless task. But even a single flawed metric beats no metrics at all.
My thinking was to just make the responses available to users and let them see how models perform. But from some feedback, turns out users don't want to have to evaluate the answers and would rather see a leaderboard and rankings.
The scalable solution to that would be LLM as judge that some benchmarks already use, but that just feels wrong to me.
LM Arena tries to solve this with the crowd sourced solution, but I think the right method would have to be domain expert human reviewers, so like Wirecutter VS IMDb, but that is expensive to pull off.
I saw a study where a prompt massively boosted one model's performance on a task, but significantly reduced another popular model's performance on the same task.
Reminder that in most cases, it's impossible to know if there is cross-contamination from the test set of the public benchmarks, as most LLMs are not truely open-source. We can't replicate them. So arguably it's worse in some cases, pretty much fraud if you account for the VC money pouring in. This is even more evident in unknown models from lesser known institutes like from UAE.
Configure aider or claude code to use the new model, try to do some work. The benchmark is pass/fail, if after a little while I feel the performance is better than the last model I was using it's a pass, otherwise it's a fail and I go back.
Building your own evaluations makes sense if you're serving an LLM up to customers and want to know how it performs, but if you are the user... use it and see how it goes. It's all subjective anyway.
I'd really caution against this approach, mainly because humans suck at removing emotions and other "human" factors when judging how well something works, but also because comparing across models gets a lot easier when you can see 77/100 vs 91/100 as a percentage score, over your own tasks that you actually use the LLMs for. Just don't share this benchmark publicly once you're using it for measurements.
At this point anyone using these LLMs every day have seen those benchmark numbers go up without an appreciable improvement in the day to day experience.
Yeah no you're right, if consistency isn't important to you as a human, then it doesn't matter. Personally, I don't trust my "humanness" and correctness is the most important thing for me when working with LLMs, so that's why my benchmarks focus on.
> At this point anyone using these LLMs every day have seen those benchmark numbers go up without an appreciable improvement in the day to day experience.
Yes, this is exactly my point. The benchmarks the makers of these LLMs seems to always provide a better and better score, yet the top scores in my own benchmarks have been more or less the same for the last 1.5 years, and I'm trying every LLM I can come across. These "the best LLM to date!" hardly ever actually is the "best available LLM", and while you could make that judgement by just playing around with LLMs, actually be able to point to specifically why that is, is something at least I find useful, YMMV.
When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much more human-like reasoning ability, than the ability to multiply large numbers or whatever (computers were already good at that, to the point that it has become a useless skill for humans to have). The point of these LLMs is to do things that computers were bad at.
However:
> Testing only on these problems would not predict performance on larger numbers, where LLMs struggle.
Since performance on large numbers is not what these exams are intended to test for, I don’t see this as a counterargument, unless the benchmarks are misrepresenting what is being tested for.
Or given a calculator. Which it's running on. Which it in some sense is. There's something deeply ironic about the fact that we have an "AI" running on the most technologically advanced calculator in the history of mankind and...it can't do basic math.
I think the tokenization is a bigger problem than the model itself.
So while it can look like a LLM calculates correctly, its still restricted by this accuracy issue. What happens when you get a single number wrong in a calculation, everything is wrong.
While a calculator does not deal with predictions but basic adding/multiplying/subtracting etc .. Things that are 100% accurate (if we not not count issues like cosmic rays hitting, failures in silica etc).
A trillion parameter model is just that, a trillion parameters, but what matter is not the tokens but the accuracy as in, the do they use int, float16, float32, float64 ... The issue is, the higher we go, the memory usage explodes.
There is no point in spending terabytes of memory, to just get a somewhat accurate predictive calculator, when we can just have the LLM call a actual calculator, to ensure its results are accurate.
Think of a LLM more like somebody with Dyslexia / Dyscalculia... It does not matter how good you are, all it takes is to switch one number in a algebraic calculation to get a 0/10 ... The reason why i mention this, is because i often think of a LLM like a person with Dyslexia / Dyscalculia. It can have insane knowledge, be smart, but be considered dumb by society because of that less then accurate prediction (or number swiping issue).
Take it from somebody that wasted a few years in school thanks to that issue, it really does not matter if your a good programmer later in life, when you flunk a few years thanks to undiagnosed issues. And yet, just like a LLM, i simply rely on tool usage to fix my inaccuracy issues. No point in wasting good shoulder space trying to graft a dozen more heads/brains onto me, when i can simply delegate the issue away. ;)
The fact that we can get computer models, that can almost program, write texts, ... and do so much more like a slightly malfunctioning human, amazes me. And at the same time, i curse at it like my teachers did, and also call it dumb at times hehehe ... I now understand how my teachers felt loool
My email client runs on my computer and it doesn’t do basic arithmetic either.
Something running on a computer does not imply that it can or should do basic arithmetic
I guarantee that computer vision and email clients both use basic arithmetic in implementation. And it would be trivially easy to bolt a calculator into an email app, because the languages used to write email apps include math features.
That's not true of LLMs. There's math at the bottom of the stack. But LLMs run as a separate closed and opaque application of a unique and self-contained type, which isn't easily extensible.
They don't include hooks into math features on the GPUs, and there's no easy way to add hooks.
If you want math, you need a separate tool call to conventional code.
IMO testing LLMs as if they "should" be able to do arithmetic is bizarre. They can't. They're not designed to. And even if they did, they'd be ridiculously inefficient at it.
I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.
That is only marginally less pedantic than saying that the only thing computer vision does is run discrete electrical signals through billions of transistors.
If allowing this behaviour raises a problem, you can always add constraints to the benchmark such as "final answer must come out under 15s" or something. The LLM can then make the decision to ask around in accordance to the time risk.
Personally, I’d say that such tool use is more akin to a human using a calculator.
People interested can see the results of giving LLMs pen and paper today by looking at benchmarks with tools enabled. It's an addition to what you said, not an attack on a portion of your comment :).
At the very least, the scores for benchmarking a human on such a test with and without tools would be different to comparing an LLM without the analogous constraints. Which is (IMO) a useful note in comparing reasoning abilities and why I thought it was interesting to note this kind of testing is just called testing with tools on the LLM side (not sure there is an equally as standard term on the human testing side? Guess the same could be used for both though).
At the same time I'm sure other reasoning tests don't gain much from/expect use of tools at all. So it wouldn't be relevant for those reasoning tests.
How so? Isn't the point of these exams to test arithmetic skills? I would hope we'd like arithmetic skills to be at a constant level regardless of the size of the number?
College exam takers use those tricks because they are on a time limit and are gaming the system. It's clever and wink wink nudge nudge ok everyone does it. But it's one tiny signal in a huge spectrum of things we use to evaluate people.
Instead, these metrics are gamed and presented as the entire multi special signal of competence for LLMs because it is literally impossible to say that success in one domain would translate the way it might with a good hire.
What I want is something I don't have to guard against gaming. Something conscientious and capable like my co workers. Until then it's google version 2 married to intellisense and I'm not letting do anything by itself.
That's a good point imo but we achieved this stuff by at least 2022 when ChatGPT was released. The thing about these giant black boxes is that they also fail to do things that directly human-written software ("computers") does easily. The inability to print text onto generated images or do general arithmetic is important. And sure, some of these limits look like "limits of humans". But it is important to avoid jumping from "they do this human-thing" to "they're like humans".
edit: I forgot my point: calculating big numbers is not a real world problem anyone has.
The way they’re being deployed it feels like the point of LLMs is largely to replace basic online search or to run your online customer support cheaply.
I’m a bit out on a limb here because this is not really my technical expertise by any stretch of the imagination, but it seems to me these benchmark tests don’t really tell us much about how LLM’s perform in the ways most people actually use them. Maybe I’m off base here though
They're still brand spanking new and everyone's trying to figure out how to best use them. We don't even really know if they're ever going to be "really good at" any given task!
Are they "really good at" these things or are they merely "OK-ish"?
* Answering factual questions.
* Programming.
* Understanding what the user wants from natural language.
* Searching/recommending stuff.
Yet literally hundreds of billions of dollars are being invested in them. That’s what’s so concerning. And I can tell you not one of these startups would EVER acknowledge the truth of your statement.
Someone want to start? I've got a Yjs/CRDT collaborative editing bug that took like a week and a half of attempts with Claude Code (Sonnet 4.5), GPT5-codex (medium), and GLM-4.6 many, many attempts to figure out. Even then they didn't really get it... Just came up with a successful workaround (which is good enough for me but still...).
Aside: You know what really moved the progress bar on finding and fixing the bug? When I had a moment of inspiration and made the frontend send all it's logs to the backend so the AIs could see what was actually happening on the frontend (near real-time). Really, I was just getting sick of manual testing and pasting the console output into the chat (LOL). Laziness FTW!
I have the Google Chrome Dev Tools MCP but for some reason it doesn't work as well :shrug:
It's more token efficient too since I don't need to load the full MCP description into my context.
Think of it as TDD.
> we (expert developers) ...
> took like a week and a half of attempts with Claude Code ...
What kind of expert developer wastes that much time prompting a bunch of different LLMs to end up with a workaround, instead of actually debugging and fixing the bug themselves?
I think any LLM-user worth their salt have been doing this pretty much since we got API access to LLMs, as otherwise there is no way to actually see if they can solve the things you care about.
The only difference is that you must keep the actual benchmarks to yourself, don't share them with anyone and even less put them publicly. The second you do, you probably should stop using it as an actual benchmark, as newly trained LLMs will either intentionally or unintentionally slurp up your benchmark and suddenly it's no longer a good indicator.
I think I personally started keeping my own test cases for benchmarking around the GPT3 launch, when it became clear the web will be effectively "poisoned" from that part on, and anything on the public internet can be slurped up by the people feeding the LLMs training data.
Once you have this up and running, you'll get a much more measured view of how well new LLMs work, and you'll quickly see that a lot of the fanfare doesn't actually hold up when testing it against your own private benchmarks. On a happier note, you'll also be surprised when a model suddenly does a lot better in a specific area that wasn't even mentioned at release, and then you could switch to it for specifically that task :)
Probably because Sonnet is no longer a frontier model, it isn't even the best model Anthropic offers, according to themselves.
Maybe we need something similar for benchmarks, and updated for today's LLMs, like:
> LLM benchmarks can be used to show what tasks they can do, but never to show what tasks they cannot.
My use-case is probably pretty far from the usual tasks: I'm currently implementing a full observability platform based on VictoriaMetrics / Victorialogs + Grafana. It's quite elaborate and has practically no overlap with the usual/cloud solutions you find out there. For example, it uses an authenticated query stack: I use the Grafana oauth token to authenticate queries by injecting matchers via prom-label-proxy and forward that to promxy for fan-out to different datasources (using the label filter to only query some datasources). The IaC stuff is also not mainstream as I'm not using any of the big cloud providers, but the provider I use nonetheless has a terraform provider.
As you can imagine, there's probably not much training data for most of this, so quality of the responses varies widely. From my experience so far Claude (Sonnet 4.5 ) does a _much_ better job than GTP-5 (Codex or normal) with the day-to-day task. Stuff like keeping documentation up to date, spotting inconsistencies, helping me find blind spots in the Alerting rules, etc. It also seems to do better working with provided documentation / links.
I've been using Claude for a couple of weeks now but recently switched to codex after my subscription to Claude ran out. I was really curious after reading a lot of good things about it but I gotta say, so far, I'm not impressed. Compared to Claude it gives wrong answers much more frequently (at least in this domain). The results it produces take much more effort to clean up than Claude's. Probably on a level where I could just invest the time myself. Might be that I do not yet know how to correctly prompt GPT but giving both tools the same prompt, Claude does a better job 90% of the time.
Anyway, I guess this is my long-winded way of saying that the quality of responses "off the beaten track" varies widely and is worth testing several models with. Especially if your work is not 70+% of coding. Even then I guess that many benchmarks have seized being useful by now?
You can get a lot of free usage out of the models.
It's primarily marketing-driven. I think the technical parts of companies need to attempt to own this more.
It gets really weird when engineering priorities shift because of these mostly irrelevant benchmarks.
> Our systematic review of 445 benchmarks reveals prevalent gaps that undermine the construct validity needed to accurately measure targeted phenomena
Intelligence has an element of creativity, and as such the true measurement would be on metrics related to novelty, meaning tasks that have very little resemblance to any other existing task. Otherwise it's hard to parse out whether it's solving problems based on pattern recognition instead of actual reasoning and understanding. In other words, "memorizing" 1000 of the same type of problem, and solving #1001 of that type is not as impressive as solving a novel problem that has never been seen before.
Of course this presents challenges to creating the tests because you have to avoid however many petabytes of training data these systems are trained with. That's where some of the illusion of intelligence arises from (illusion not because it's artificial, since there's no reason to think the brain algorithms cannot be recreated in software).
I’m not sure what your basis is for saying “LLMs fail if there is a small wrench in the prompt.” They also succeed despite wrenches in the prompt with great regularity.
It's not just semantics, the metrics are supposed to tell us the potential of the model. If they can solve extremely hard PhD problems, it should be the case that we're already in the singularity, and they should be solving absolutely everything in whatever field they were trained in, because it's not just PhD level, it's a machine that has a ton of memory, compute and never sleeps. However, once you use these models extensively, it becomes apparent they are just synthesizing data, and not as much understanding it in a way that would allow them to extrapolate into anything else as humans do.
I think this point is a little hard to explain. I'll just emphasize, these are smart systems, and they can do a lot, but there is still a disconnect between, let's say, a PhD level model and a human with a PhD, in the "quality" of what we would call "intelligence" of both entities (human and machine).
Someone describing string theory is the literary equivalent of fractal structures in snowflakes. Lovely, complex, possibly unique, but not proof of a level of intelligence- for the string theorist maybe it is intelligent, perhaps persuading someone to fund their grant, which enables them to eat, shelter etc. Might be a bit harsh on string theory. Saying it is proof of an amount of intelligence leads us to falsifiable statements.
The big difference from LLMs is that we don’t really have production-grade, standardized benchmarks for long-form TTS. We need things like volume-stability across segments, speech-rate consistency, and pronunciation accuracy over a hard corpus.
I wrote up what this could look like here: https://lielvilla.com/blog/death-of-demo/
if a single benchmark could be a universal truth, and it was easy to figure out how to do it, everyone would love that.. but that's why we're in the state we're in right now
Even if it requires human evaluators at first, and even if the models completely suck at this task right now: it seems like the kind of task you'd want them to be good at, if you want these models to eventually carry out these tasks in embodied forms in the real world.
Just having the benchmark in the first place is what gives model makers something to optimize for.
I can imagine a robotics architecture where you have one model generating footage (next frames for what it is currently seeing) and another dumber model which takes in the generated footage and only knows how to generate the motor/servo control outputs needed to control whatever robot platform it is integrated with.
I think that kind of architecture decoupling would be nice. It allows the model with all the world and task-specific knowledge to be agnostic from its underlying robot platform.
The upside is that these can all be generated and checked synthetically so large data sets are possible, in both formal and natural languages.
Benchmarks make for a good first pass though to figure out which ones to test
(Don't hurt me, I just like his chatbot. It's the best I've tried at, "Find the passage in X that reminded me of the passage in Y given this that and the other thing." It has a tendency to blow smoke if you let it, but they all seek to affirm more than I'd like, but ain't that the modern world? It can also be hilariously funny in surprisingly apt ways.)
"For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle."
At least some of those who design benchmark tests are aware of these concerns.
If anyone doubts my concerns and thinks this article is in good faith, just check out this site's "AI+ML" section: https://www.theregister.com/software/ai_ml/
https://openreview.net/pdf?id=mdA5lVvNcU
And the review is pretty damning regarding statistical validity of LLM benchmarks.
AI (and humans!) aside, claiming that there was an oracle that could "answer all questions" is a solved problem. Such a thing cannot exist.
But this is going already too deep IMO.
When people start talking about percentages or benchmark scores, there has to be some denominator.
And there can be no bias-free such denominator for
- trivia questions
- mathematical questions (oh, maybe I'm wrong here, intuitively I'd say it's impossible for various reasons: varying "hardness", undecidable problems etc)
- historical or policital questions
I wanted to include "software development tasks", but it would be a distraction. Maybe there will be a good benchmark for this, I'm aware there are plenty already. Maybe AI will be capable to be a better software developer than me in some capacity, so I don't want to include this part here. That also maps pretty well to "the better the problem description, the better the output", which doesn't seem to work so neatly with the other categories of tasks and questions.
Even if the whole body of questions/tasks/prompts would be very constrained and cover only a single domain, it seems impossible to guarantee that such benchmark is "bias-free" (I know AGI folks love this word).
Maybe in some interesting special cases? For example, very constrained and clearly defined classes of questions, at which point, the "language" part of LLMs seems to become less important and more of a distraction. Sure, AI is not just LLMs, and LLMs are not just assistants, and Neural Networks are not just LLMs...
There the problem begins to be honest: I don't even know how to align the "benchmark" claims with the kind of AI they are examinin and the ones I know exist.
Sure it's possible to benchmark how well an AI decides whether, for example, a picture shows a rabbit. Even then: for some pictures, it's gotta be undecidable, no matter how good the training data is?
I'm just a complete layman and commenting about this; I'm not even fluent in the absolute basics of artificial neural networks like perceptrons, gradient descent, backpropagation and typical non-LLM CNNs that are used today, GANs etc.
I am and was impressed by AI and deep learning, but to this day I am thorougly disappointed by the hubris of snakeoil salespeople who think it's valuable and meaningful to "benchmark" machines on "general reasoning".
I mean, it's already a thing in humans. There are IQ tests for the non-trivia parts. And even these have plenty of discussion revolving around them, for good reason.
Is there some "AI benchmark" that exclusively focuses on doing recent IQ tests on models, preferably editions that were published after the particular knowledge cutoff of the respective models? I found (for example) this study [1], but to be honest, I'm not the kind of person who is able to get the core insights presented in such a paper by skimming through it.
Because I think there are impressive results, it's just becomimg very hard to see through the bullshit at as an average person.
I would also love to understand mroe about the current state of the research on the "LLMs as compression" topic [2][3].
[1] https://arxiv.org/pdf/2507.20208
What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?
Nothing says confidence that AGI is imminent like needing the US government to prevent your investments from losing you money.
Can you link some of these comments you consider useful but got flagged?