Talk Is Cheap: The Operational Impact of LLM Use | Hacker News Reader

Talk Is Cheap: The Operational Impact of LLM Use

unessays.substack.com

35 pointsoudlys23 comments

Hah, since it's open in another tab: Talk Isn’t Always Cheap: Understanding Failure Modes in Multi-Agent Debate @ https://arxiv.org/html/2509.05396v2

To actually engage more with the substance of TFA.. very refreshing to see someone bringing numbers. To me this shows we (still) need something somewhere between the numbers and the anecdata. It's annoying to hear claims of "1000x productivity" or claims of negative/neutral productivity without any extra context whatsoever. So you brought data? Great! But still no context. Boring CRUD? Complex UI? A rewrite/port of legacy? What industry, language, how many human collaborators, and what baseline for SLOC??

We need to get rigorous about this stuff and actually aim for a decent framework which can answer "Are LLMs a value add for this project? How much value? How much cost?"

Such a thing might be information-theoretical, complexity-based, or counting integration touchpoints / info boundaries / sources of ground-truth? We could even try to implement that framework with LLMs and probably should! But the default answer of "Yes definitely, always useful, just token-maxx it since LLMs are the future" is (still) only marketing, not engineering

More tasks get "done" while rework is sky high and overall throughput to production drops.

First, I'd like to thank all the people working on testing and doing the lord's work.

Anyway, this isn't even a unique pattern to LLM use. We've all seen this exact same thing when more devs are added a project running late, teams are siloed, outsourcing to contractors, etc.

Definitely not unique. Thats said, given how "easy" it is to scale LLM output, compared to human output, this pattern could be messier in the LLM era? The whole tokenmaxxing motion assumed more output equals more outcomes. What do you think?

Already covered in the post.

> So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution on. They never can be.

Fair. I suppose, the question behind my question is -- what else do we need. Thanks.

Yeah I have a different take. We will end up scaling human output by hiring more devs as wages for entry to mid level continue to stagnate.

I think LLMs make finding information and learning way more accessible. Even with realistic expectations it already is a revolution for literacy, education, and search. LLMs are a massive achievement that would be celebrated appropriately if only the public wasn't introduced to them during global political/economic crises encouraging grifters and authoritarians muddying the waters.

To be clear, hiring at scale is a sorely needed step. Software is just like any other form of writing. It carries weight and needs cultural and community context to work. We have needed way more technical literacy for decades to make this digital always-on world work. I think most would at least agree that LLMs bridge knowledge gaps. They're a net good thing despite the current abuse by bad actors.

Looking forward to that hiring. So far, just layoffs. That said, there reports that consulting firms are hiring to fuel the demand around AI adoption itself. LLMs are a breakthrough like machines were. We will create new business models around them, and perhaps drives more humans. Wrt software, LLMs can make learning very easy. But with all the AI generated code, who is the "architect"?

Given the last few years, I don't see why traditional software and app dev jobs would be impacted. The current architect is still the architect.

New jobs are new jobs. The catch is these new jobs that lean heavily on AI will be a race to the bottom regarding pay. There are so many people with their own special interests that thus far never had a chance to contribute without coding skills. I don't mean that in a sinister way, but it makes total sense that we'd continue to see the rest of society find their way into the decision making. Right now we have a ton of lower priority stuff that never gets implemented because it's not part of the MVP. We also have codebases not as modular and robust as they should be behaving as platforms.

It also makes total sense this is a nightmare for those people who were most loudly proclaiming we'd replace white collar jobs. They showed their cards too early and will have the mob coming after them.

Code is meant to be shared and collaborated on. That was always where the value came from. There's a lot of untapped potential when truly anyone can write their plugins and build communities. This is really is like the early internet era repeating itself, but reaching way deeper this time.

I thought this was the weakest argument in the article. Yes, non-determinism is inherent to LLMs. It's also inherent to human brains, yet somehow we still manage to call (some) people reliable.

The bit about backwards compatibility doesn't make sense either. LLMs, just like humans, can take backwards compatibility into consideration — or not. Sometimes they'll forget, and sometimes they'll fixate on it even in a system that hasn't been deployed yet. It has nothing to do with LLMs never doing "the same thing the same way twice".

LLMs may or may not become as reliable as other productivity-improving infrastructure, but it's not the nature of their technology that will decide this.

Human brains can explain themselves. What's a black box about LLMs is that they can't. They don't know how they arrived. They didn't arrive at anything. They don't have access to their internal representations. They don't think.

I think the real bitter lesson is nine women will never make a baby in a month.

Is it bitter or is it beautiful?

The idea that speed is the only thing that matters is a modern view that you can surrender any time you want.

A child is a joy. Nothing is wrong with the child for needing 9 months to grow.

I'm curious, LLMs have been around for a while now...

How many of you would say you need LLMs now for work? Not that you want it because it's nice to have, but rather you would literally not be able to do your job at all because you don't have an LLM to use?

If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go back to writing everything by hand?

I feel like companies could definitely just push the cost of LLMs back onto the engineers themselves (much like how people have to pay for their own gas to go to work), and engineers would have no choice but to either buy their own subscriptions or be very good at writing code by hand just to stay competitive.

This kind of shift is coming, partly because costs of LLMs are to unsustainable for companies, but also because it sounds like the kind of diabolical idea some upper management thinks they can get away with, as peer pressure will naturally do its thing. Paying for your own token usage is a small price to pay for job security isn't it?

I'm an embedded systems developer. I have almost fully "outsourced" the Python code for frontend pc software that interacts with my firmware.

I deliberately continue to write all my firmware by hand, and will occasionally consult AI for review. I never use AI to write prose for me.

Python is better represented in training data, writing bench software was a bit boring, I get to spend more time where I have (and continue to build) domain knowledge.

Agentic Opus is a nice to have and I get to explore the frontier tech, but if (or when) it's taken away, a self hosted coding model would be fine - I'd just have to dust off my Python skills and it would take longer.

> If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go back to writing everything by hand?

I already write code myself, so perhaps I'm not your target audience. But I strongly believe that anyone who pays for his own LLM is being a fool. You should never pay for tools for work. If you do so, you're letting them take advantage of you rather than paying the needs of their own business. It's a bad deal, don't go down that road.

But you don’t need an LLM.

“It’s hard for me to put into words how bad this is.”

AHAHAHAHA! I genuinely laughed out loud, filled the room.

This is the “citation needed” rebuttal most REASONABLE people in the software industry have been looking for, and the sample size is only going to get bigger! Does anyone really think - not believe, logically conclude based on evidence at hand - that there will be any contrary outcomes at scale? Honestly, this is going to put a lot of software pros into a “sabbatical” where cleaning up the mess should be billed like that old joke at auto mechanics’ shops:

$150 an hour to fix it

$500 an hour if you tried to fix it before bringing it here

Seriously laughing out loud. I needed this. Hahaha

Edit: no wonder he’s so astute - came up in construction. In that industry cost versus benefit and safety concerns are often a matter of life or death. Real consequences. A lot more educational than staring at a glowing box for most of one’s career. Disclosure: I did risk management for construction projects, like the Alabama MB plant.

The activity-vs-outcome distinction feels like the most interesting part of this discussion. Curious what metrics people have found most useful for separating the two.

This effect is likely be excarcebated in teams which are suffering layoffs this year. Even less people to clear the in-flight queue of partially complete tasks

People are still figuring things out, there's a lot of wasted tokens, etc.

This is like complaining a student isn't as productive as a senior engineering.

I think we as an industry haven't even graduated to junior level when it comes to figuring our how to use AI to improve things.

This is discussed in the article, and I think the author makes pretty reasonable arguments for why by nature we will not see the reliability of LLM usage improve. They also discuss what I agree as the more effective method of using an LLM is, as a feedback and refinement tool, not a decision maker.

> This is not a limitation that can be overcome by LLMs. Their generative value is in their unreliability. If you turn temperature down to zero, you get a deterministic machine - but you also break every meaningful application I know of in production.

This is not a reasonable argument. Setting the temperature down to zero does NOT give you a deterministic machine. And I have never seen that break any application in production, quite the contrary.

By “break” I don’t mean “won’t function”. I mean “won’t deliver on their value proposition”. A functioning product is a necessary, but not a sufficient condition for technology to have utility. I would defend vigorously that the generative value in LLMs is derived from their unreliability. Which is what the argument ultimately rests on.

Reply on news.ycombinator.com