LLM Inflation
tratt.net
tratt.net
> Bob’s manager receives 4 paragraphs of dense prose and realises from the first line that he’s going to have to read the whole thing carefully to work out what he’s being asked for and why. Instead, he copies the email into the LLM.... The 4 paragraphs are summarised as “The sender needs a new computer as his current one is old and slow and makes him unproductive.” The manager approves the request.
"LLM inflation" as a "bad" thing often reflects a "bad" system.
In the case described, the bad system is the expectation that one has to write, or is more likely to obtain a favorable result from writing, a 4 paragraph business case. Since Bob inflates his words to fill 4 paragraphs and the manager deflates them to summarise, it's clear that the 4 paragraph expectation/incentive is the "bad" thing here.
This phenomenon of assigning the cause of "bad" things to LLMs is pretty rife.
In fact, one could say that the LLM is optimizing given the system requirement: it's a lot easier to get around this bad framework.
At the NYC employment job, I was on a 2 year upgrade cycle being a software developer. Just whatever the current year Dell corpo laptop was. No procurement procedures, just IT got 2 year replacements, our laptops went to the non IT workforce after reimaging.
As a consultant I usually bring my own device, and my laptops are usually WAY more capable since I will run each client on their own VM - makes it easy to delete the client when the contract is up. But I've had one client who did not allow BYOD, and any billed work had to be done on their hardware. That was fine, except that the desktop I was given was already a 12 year old dual core non-hyperthreading CPU that wasn't meant for developers even when it was built. I begged and pleaded for 6+ months for me to either bring in my own hardware they could image now and wipe at the end of the contract, or to please buy me a PC from this decade.
It took 3 years to get the budget approval for a $2000 tower, roughly the equivalent of 15 hours of pay. The thing that finally pushed it over the edge, was that my PC could not handle Teams + Visual Studio at the same time, and manager couldn't handle that he couldn't watch me program.
All of that to say I doubt these non-data-driven organizations are basing these decisions on anything other than micromanagement. Nothing to do with measured or assumed productivity, nothing to do with costs, so all I can think is they have to be a "decision maker" on all aspects.
As to the content of the letter, the 4 paragraphs are supposed to be "these are reasons I think were missed and why it'll cost more to not correct it" not just "I put effort to write 4 paragraphs of stuff" friction alone.
Having run a short stint as an internal IT manager at an IT focused company... it's astounding how many non-standard/out-of-cycle laptop request are actually either basic user error (even for the most brilliant technical employees) or basic IT systems problems (e.g. poorly tested management/security tool changes eating up performance/battery in certain configurations) that a new laptop won't actually solve. E.g. reports of "my battery runs out in 2 hours and my IM is dog slow" but they are on an M1/M2 MacBook Pro and probably wouldn't notice if they got an M1 or M4 MacBook back as their issue isn't actually the hardware. When someone writes an email or ticket explaining why their use case just wasn't accounted for it's generally pretty obvious they really do need something different.
This might have been a genuinely useful system, something which broke down with the existence of LLMs.
Social media will give you a good idea what sort of person enjoys writing 4 paragraphs when something goes wrong; do you really want to incentivize that?
Gen-AI completely negates meaningless verbosity as a proxy of time spent. It will be interesting to see what emerges as a new proxy, since time-as-currency is extremely engrained into the fabric of human social interactions.
There are some parallels to that in compression and cryptography, but they are rather far-fetched.
If someone wants a new computer they should just have to say why. And if it's a good reason, give it to them. If not, don't. Managers have to manage. They have to do their jobs. I'm a manager and I do my job by listening to the people I manage. I don't put them through humiliation rituals to get new equipment.
People want new computer because new hire Peter got a new one.
People want new computer because they just had lunch with a friend that works in a different company and got a new computer and they just need to one up him next time they go to lunch.
That is why I am not going to just give people computers because they ask. Worst crybabies come back because they „spilled coffee” on perfectly fine 2 years old laptop.
>If someone wants a new computer they should just have to say why. And if it's a good reason, give it to them. If not, don't.
Not everything you read on the internet is 1 to 1 real life ;)
If you are awarding computers based on 3 paragraph essays, you are having horribly inefficient process that rewards creative writing rather then work.
Which is why none of it happen in real companies, unless they are someones startup expected to fail anyway.
so I'm sure there's large corps that do this for everything. probably ones where you're not asking your manager, but asking finance or IT for everything
If they don’t care they don’t care. They pay most of us for our time anyway, not what we achieve.
Sure, as long as we completely disregard the water, power and silicon wasted to accomplish this goal.
I've been aware of similar dynamic in politics, where the collective action/intelligence of the internet destroyed all the old signals politicians used to rely on. Emails don't mean anything like letters used to mean. Even phone calls are automated now. Your words and experience matter more in a statistical big data sense, rather than individually.
---
This puts me in sci-fi world-building mode, wondering what the absurd extension is... maybe it's just proving burned time investment. So maybe in an imagined world where LLMs are available to all as extensions of thought via neural implant, you can't be taken seriously for even the simplest direct statements unless you prove your mind sat and did nothing (aka wasted it's time) for some arbitrary period of time. So if you sat in the corner and registered inactive boredom for 2h, and attached a non-renewable proof of that to a written word, then people would take your perspective seriously, because you expended (though not "gave") your limited attention/time to the request for some significant amount of time
Because politicians literally write the rules of the system, it's incredibly difficult to prevent abuse, bad incentives, and inefficiency.
One of the most fundamentally broken aspects of the US system is that:
1. politicians stay in power by being elected.
2. people are not required to vote and casting a vote incurs a cost (time, travel, expense), therefore not everyone votes.
3. politicians just have to get the votes of the fraction of people who actually do vote and can ignore those who don't.
What may seem like abuse and bad incentives to YOU are perfectly aligned goals for THEM.
Want a bandaid for people voting randomly? Throw in a randomly (llm?) generated question about the policy they're voting on and if they get it correct, showing some rudimentary understanding of the policy, they can vote.
Bob’s manager is lazy and or an idiot.
Probably both.
1) For pedagogical or explanatory purposes. For example, if I were to write:
> ∀x∈R,x^2≥0
I've used 10 characters to say
> For every real number x, it's square is greater than or equal to zero
For a mathematician, the first is sufficient. For someone learning, the second might be better (and perhaps as expansion of 'real number' or that 'square' is 'multiplying it by itself').
2) To make sure everything is stated and explicit. "He finally did x" implies that something has been anticipated/worked on for awhile, but "after a period of anticipation he did x" makes it more clear. This also raises the question of who was anticipating, which could be made explicit too.
As someone who spends a lot of time converting specifications to code (and explaining technical problems to non-technical people), unstated assumptions are very prevalent. And then sometimes people have different conceptions of the unstated assumption (i.e. some people might think that nobody was anticipating, it just took longer than you'd expect otherwise).
So longer text might seem like a simple expansion, but then it ends up adding detail.
I definitely agree with the authors point, I just want to argue that having a text-expander tool isn't quite as useless as 'generate garbage for me'.
The idea that generators are some sort of parrot is very outdated. The 2021 paper that coined the term "stochastic parrot" was already wrong when it was published.
Sure. But can they read the original author's mind, and therefore generate the right unique sentence that expresses the actual intent?
obviously it can generate the longer message, but is it going to go look up what the sentence refers to and infer extra meaning automatically...?
It still surprises me when I see non-technical enthusiasts get excited about LLMs drafting almost useless copy or email or whatever. So much garbage text no one reads but has to be written for some reason. Its weird.
When writing something I want people to read, I always take time at the end to make it shorter - remove distracting sentences, unnecessary adjectives and other noise. Really works wonders for team communication.
This is a good advice. How can I do it when talking? I often talk too much saying little, often loosing Listener's attention in the process.
Silence is better than useless noise.
What incredible technology.
You are so right! LLMs produce so much noise. If you ask them to be concise, they struggle to cut just the fat, and the output is often vague or misleading. I see that again and again when I ask it to produce different versions of a sentence.
I imagine it's how artists feel about AI art. It seems right at first glance, but you can tell that no thought or craftsmanship went into it.
Inference is then the decompression stage where it generates text from the input prompt and the compressed model.
Now that compressing and decompressing texts is trivial with LLMs, we humans should focus - in business at least - on communicating only the core of what we want to say.
If the argument to get a new keyboard is: "i like it", then this should suffice, for inflated versions of this argument can be trivially generated.
What a horrible technology.
Maybe you can provide an example where this case would occur, and maybe some indication how often you think this would occur.
https://royalsocietypublishing.org/doi/10.1098/rsos.241776
Edit: I was thinking about the “overly snide” and am reminded of Sam Bankman-Fried:
I would never read a book. I think, if you wrote a book, you fucked up, and it should have been a six paragraph blog post.
This seems like exactly what LLMs are supposed to be good at, according to you, so why don't they just near-losslessly compress the data first, and then train on that?
Also, if they're so good at this, then why are their answers often long-winded and require so much skimming to get what I want?
I'm skeptical LLMs are accurately described as "near lossless de/compression engines".
If you change the temperature settings, they can get quite creative.
They are their algorithm, run on their inputs, which can be roughly described as a form of compression, but it's unlike the main forms of compression we think of - and it at least appears to have emergent decompression properties we aren't used to.
If you up the lossy-ness on a JPEG, you don't really end up with creative outputs. Maybe you do by coincidence, and maybe you only do with LLMs - but at much higher rates.
Whatever is happening does not seem to be what I think people typically associate with simple de/compression.
Theoretically, you can train an LLM on all of Physics, except a few things, and it could discover the missing pieces through reasoning.
Yeah, maybe a JPEG could, too, but the odds of that seem astronomically lower.
And if you find a way to compress text that isn't insanely computationally expensive, and still makes the compressed text compressible by LLMs further - i.e. usable in training/inference? You, basically, would have invented a better tokenizer.
A lot of people in the industry are itching for a better tokenizer, so feel free to try.
If trust was higher, shorter documents would be more desirable. If trust was lower, or accountability higher, summarization would be used a lot more carefully.
LLMs haven't changed anything in this regard except that they've made it extremely easy to abuse trust at that specific level. The long-term result will be that trust will fall in the general case, and people will eventually become more careful about using summarization. I don't think it will be long before productized AI used in business contexts will be pretrained/fine-tuned to perform a basic level of AI content detection or include a qualitative measure of information density by default when performing summarization.
Yeah, this is the problem. Wealth distribution stopped working sometime in the late 20th century and we're fighting each others for competitive advantages. That's the core of this phenomenon.
No one needs containers full of baby sized left shoes, but proof of work must be shown. So the leathers must be cut and shoes must be sewn, only to be left in the ever growing pile in the backyard. That's kind of wrong.
There's a line of thought which states that intelligence rhymes with compression: Identifying patterns allows better prediction, enables better compression of the data.
However, internally, LLMs typically do the opposite: Tokenization and vectorization multiply the bit rate of the input signal. Chain of thought techniques add a lot of extra text, further increasing the bit rate.
Of course when I went to read them they were 100% slop. The funniest requirement were progress bars for actions that don’t have progress. The tickets were, even if you assume the requirements weren’t slop, at least 15 points a piece.
But ok maybe with all of these new tools we can respond by implementing these insane requirements. The real problem is what this article is discussing. Each ticket was also 500-700 words. Requirements that boil down to a single if statement were described in prose. While this is hilarious the problem is it makes them harder to understand.
I tried to explain this and they just said “ok fine rewrite them then”. Which I did in maybe 15min because there wasn’t actually much to write.
At this point I’m at a loss for how to even work with people that are so convinced these things will save time because they look at the volume of the output.
But project plan dates have always been fiction. Getting there faster is an efficiency win.
That said I’ve found that llms are good as interrogators. If used to guide a conversation, research background information and then be explicitly told to tersely outline the steps in something I’ve had very good results.
The same way, presumably, that one used to work with people who would say things like "just look how much code this template system generates for us!" unironically.
isn't this the opposite? Enabling compression will INCREASE the load on your server as you need more CPU to compress/decompress the data.
Humans, the only extant example of a general intelligence, don't do lossless compression at all.
I don't think you get to AGI by trying to compress noise.
Prediction is formally equivalent to compression, so loss is just a measure of how well you can compress the training dataset.
This concept probably applies to lots of work in the "AI" space right now. The idea of using huge amounts of compute to generate lifelike voices for LLMs comes to mind as being recently maligned (something many users may not want). Or people upset about getting AI summaries in search that they didn't ask for. And yet, swaths of capital has been invested in these ideas and perhaps its a worthwhile use of resources. I am not sure personally. Time will tell. But I suspect its more complicated than the author is implying here.
And if an LLM is also used at the other endpoint to parse the longer text, that creates a broken telephone. Congrats, your communication channel is now unreliable.
instead of elongated sentences, we perhaps might start seeing an increase in just communicating through the minimum constructing points of whatever meaning we hope to convey, leaving the presentation work for the LLM on the receiving side
Is this situation in any way realistic one? Because the way companies work in my beck of woods, no one wants your 4 paragraph business case essay about computer. Like, it is funny anecdote.
But, in real world, at least in my experience, pretty much everyone preferred short for emails and messages. They would skim the long ones at best, especially in situation that can be boiled down to "Tom wants a new computer and is verbose about it".
I am open to the idea that there is some bureaucratic workplace where it works like that ... but everywhere I have experience with, they preferred the short version.
I have, though not directly but seeing it in an onsite visit to a client.
It isn't as common as people complaining or joking about it might suggest, but it certainly happens.
No, it's much worse than that. In real life you talk about pages and pages of documents and power points and meetings after meetings if you happen to need a computer/server/configuration that's not in the pre-approved list. (I really wish I was exaggerating. And of course no, not all employers are like this to state the obligatory obvious.)
This of course now gets weird with LLMs because I doubt it can last as a signal of respect for very long when it just means you fed some bullet points to ChatGPT.
There might be something else at play there. Public sector workers are notorious for wooden language.
The example I hate the most is how they always say "number of" before every number like "we'll buy 10 new busses" becomes "we will commence the procurement of a number of 10 new buses".
>Bob’s manager receives 4 paragraphs of dense prose and realises from the first line that he’s going to have to read the whole thing carefully to work out what he’s being asked for and why. Instead, he copies the email into the LLM du jour and types at the start “Please summarise this email for me in one sentence”. The 4 paragraphs are summarised as “The sender needs a new computer as his current one is old and slow and makes him unproductive.”
Sam Altman actually had a concise tweet about this blog's topic (https://x.com/sama/status/1631394688384270336)
>something very strange about people writing bullet points, having ChatGPT expand it to a polite email, sending it, and the sender using ChatGPT to condense it into the key bullet points 2:42 PM · Mar 2, 2023 · 1.2M Views
Now that decorating any message with such fluff is automated, we can as well drop the requirement and just state clearly what we want without fluff.
However, you can't do much of anything to deal with people who stop reading at the first sentence or question. Those folks are hopeless.
The 4 paragraphs requirement was not introduced 'because LLM'. It was there all along for what just should have been 'gimme 2 -3 bullet points'. They wanted Bob to hold back on requesting the new machine he needed, not by denying his request openly, but by making the process convoluted. Now Bob can cut through the BS, they want to blame the LMM for wasting their time and resources? BS!
Furthermore, the guy who doesn't need a new computer is probably going to be writing the paragraphs on company time, so he'd do it if he has nothing to do and needs to look busy. (I think it's safe to assume that the sort of place where you need 4 paragraphs to justify a new computer would also require for employees to look busy at all times.)
Why is everyone trying so hard to find purpose in broken administrative processes? There's usually none and, if there is its usually so hideous that it can't be put into writing.
Broken processes persist when they're benefiting someone somehow. If you can't figure out who's benefiting from a crap process, good luck trying to fix it or change it.
But broken vestiges of long forgotten and obsoleted requirements can persist for a very long time.
Besides, inthe now wether someting will persist is mostly irrelevant for the immediate subjection to the process.
Most people don't think very clearly. That's why rhetoric is effective. That's why most communication is fluffy social signaling. You can give people great advice and their eyes glaze over because the words didn't fill them with emotion, or something, and they do the exact opposite.
No wonder LLMs get put to work playing that stupid game.
I expect smaller models to become incrementally better at compressing what truly matters in terms of information. Books, reports, blog posts… all kinds of long-form content can be synthesized in just a few words or pages. It’s no wonder that even small LLMs can provide accurate results for many queries.
What a depressing belief. Human communication is about a whole lot more than just getting your point across as quickly and efficiently as possible.
What I don’t agree with is being needlessly verbose in circumstances in which the opposite is more valuable. Unfortunately, humans have a tendency to use flowery language even when that comes at the expense of message clarity.
Think, for example, of the countless self-help books that can be converted to short blog posts. Or think about legal or academic writing, which often stands in the way of readers actually understanding what is being said.
There’s so much writing like this out there that even LLMs were notorious for taking this over-elaborate language to a whole new level. And in my opinion that’s the kind of thing that we can (and should) avoid.
I've also noticed it when people post LLM writing on Reddit. Something may give me pause, and then re-reading the content any given paragraph was way off. I had even glossed over the bolded conclusion "it’s a coaching-wheels moment" (?) because as you read it your brain thinks of a way it could make sense.
https://en.wikipedia.org/wiki/Deflate seems relevant here.
Sure, it doesn't directly capture the compression/decompression aspect, but it's assumed that slop includes unnecessary filler.
I actually straight up reject it when text is too inflated, and I remind people that LLMs are available to expand on request.
Sounds like a fun place :-)
(yes, yes, I know it's a typo, I could not resist)
My former (obviously) wannabe manager used GAI to pimp our CV's before sending out to clients, pretty sure they too consulted stupid to summarize on their end.
This is one example of the "horseless carriage" AI solutions. I've begun questioning further that actually we're going into a generation where a lot of the things we are doing now are not even necessary.
I'll give you one more example. The whole "Office" stack of ["Word", "Excel", "Powerpoint"] can also go away. But we still use it because change is hard.
Answer me this question. In the near future if we could have LLMs that can traverse to massive amount of data why do we need to make excel sheets anymore? Will we as a society continue to make excel spreadsheets because we want the insights the sheet provides or do we make excel sheets to make excel sheets.
The current generation of LLM products I find are horseless carriages. Why would you need agents to make spreadsheets when you should just be able to ask the agent to give you answers you are looking for from the spreadsheet.
Until there is a fix for this (not clear there ever will be), Excel will be necessary.
Word will probably become a different, more collaborative product. Notion-esque.
Powerpoint...I would love if it disappeared but ultimately if you have to present something, you need to have done the work.
Because it seems to be a fundamental property of LLMs that they just make things up all the time. It's better to make the LLM a natural interface to a formal query language which will return hard answers with fidelity from the database.
Think of them as an artifact of a snapshot of time. You can sign them and file them away, perform backups on them, and use that document the intent at that time.
Audits don't work so well on LLMs
A couple of related questions- if airplanes can fly themselves with auto-pilot, why do we need steering yolks? If I have a dishwasher- why do I still keep sponges and dish soap next to my sink?
The technology is nowhere near being reliable enough that we can eschew traditional means of interacting with data. That doesn't prevent the technology from being massively useful.
Now that pachinko machines can create lots of prose, maybe it's time to finally learn this lesson.
Ah, yes. It is an achievement in signals in a way.