only as long as you're trying to replace a human's job. because human jobs are structured to do a wide variety of things.
a useful agent needs a wide variety of inputs, and one single restricted action it can take. it doesn't need permission to do everything, it need permission to do the tiniest possible useful thing it can do, and nothing else.
If we are talking about human labor, how many people hack their way through their work day?
Define hack...
Not doing their work, copying other peoples work, putting off work till later, taking credit for other peoples work, literal law violations.
Actually humans do this quite a lot and there are just massive numbers of business and regulatory processes and checks to ensure they are not doing it. With humans every human that is good enough to hire and do you work you want also have the ability to steal everything in sight and run away if they so choose.
Even very llm-pilled coders i know sometimes back away from the “smartest” models, since they aren’t always better at the job at hand, and definitely not when you account for cost.
Based on my experience with running models locally, there is a threshold of intelligence required to be useful. But it’s possible there is also a ceiling where smarter isn’t necessarily better. If you ask a 4B parameter model to fix a bug, it might e.g. fix the bug but fail to fix a compilation error created by the fix. If you ask a frontier model, it might fix the bug, re-write your unit tests, and update the readme. Maybe you wanted those things but maybe you didn’t. “Smarter” is often shorthand for more proactive, and guessing more about your intent. Which is great when it gets it right, and annoying when it gets it wrong.
I suspect smaller models, tuned to a specific task, will do a VAST majority of the llm jobs. High capability huge models will be what humans want to interact with, the bare minimum that gets the job done will be everything else.
Do you want fable for one-shotting a game or website? Probably! The whole thing is mostly existing examples with small modifications that it will definitely get right. Do you want fable to just go nuts on a large, custom, unusual code base built around domain-specific problem solutions? Absolutely not, it will fix every problem it's presented with while creating lots of new ones.
Past 10k lines on something custom and with real-world complexity, you have to start thinking about which model should design, which should implement, which should review, and the appropriate effort-settings for each. Even then.. the answers aren't static because it depends on the task. And all this is assuming the starting place actually inherited reasonable due diligence on architecture/design. The idea of releasing the most generally intelligent models on 10k lines that were themselves the product of agents is yet another matter.
Part of what's at work here is that, like humans, every model can very easily create working code that it is completely incapable of maintaining. So realistically using multiple strengths tactically to avoid "excess creativity" needs to be SOP already, even if granular experts and specialists aren't in the usual workflow yet.
Productivity gains are still enormous compared to what we used to do before agents. But, I know that people don't want to stop there.
Humans can be tricked by humans too but humans care about their reputation in their communities, and at least fear from punishment.
Unless something is in the structure that makes individual choice and responsibility a meaningful source of friction and reduced velocity, it has no real impact on how AI is being used.
Depends on who you ask I guess
https://www.theregister.com/software/2026/01/15/ai-is-everyw...
https://www.zdnet.com/article/workslop-can-kill-your-product...
https://fortune.com/2026/08/22/executives-ai-productivity-la...
It takes time for decades old ways to change.
My own productivity yes but if I step back and look at a company scale, the productivity gains has been negative for our company as a data point.
We are now shipping less and with a lower quality.
I believe the lower quality but is quality so bad you are afraid to ship it now?
I guess I won't deny that we've had more build breakages than usual over the past couple months but they get resolved super quick. We don't hesitate to roll broken code back if something manages to slip in.
It’s not an issue of only more generation, it’s an issue of how much generation outstrips capacity to verify generated content.
Unlike spam, you can’t filter out and bin the stuff a colleague is sending you.
So individual productivity is up, while the costs of checking and processing generated content shifted to the rest of the org.
2 years ago I saw it, back before agents were really a thing. But I'm not sure that was an enormous productivity improvement, especially compared to agents today. As for today, everyone I talk to is some level of blindly trusting what the agent is doing or not using agents. I haven't met people in the middle ground and suspect that they are rare enough we don't really know what their productivity gains are.
Human in the loop has become a convenient security-theater-washing for agentic AI.
Outside of coding, I think the issue is even worse because humans will defer so much judgment to AI that the same would apply. Look at how much trouble we've had before modern AI where humans blindly trusted the computer's output rather than make their own judgment, even if their job was to be providing a safeguard against the computer's judgment.
Semi-automation (human in the loop) can still result in a dramatic uplift in productivity. You can't run a combine harvester 100% autonomous but that doesn't stop anyone from trying to get as close to that limit as possible.
I'm curious why you think that.
Refueling? Seems solvable. Tornadoes? Not directly solvable, but, no less so than for humans.
There’s infinite complexity, sure, but that’s why it’s silly to try and hop to done. One step at a time.
One step at a time is what is happening and the improvement and rate of improvement is crazy as we see,
A whole class of nontechnical people don’t accept anything but “fully solved including every possible edge case” before they call it done, then complain that they didn’t prepare socially for what happens when that is true.
Similar to problem to how 100% autonomous vehicles don’t exist, yet. There are too many edge cases.
Get to 99% first.
Precision Agriculture stuff is utterly crazy these days, other than fuel the remaining staff is the only thing left where you can get efficiency improvements - and at the scale of modern megafarms, even small percentages add up to a ton of money.
You essentially said: you’re wrong, it is autonomous when it doesn’t need a human during one specific part of its overall usage.
During the time that actually matters economically. The time to drive the harvester to/from your typical US mega-field is minuscule compared to the time it can run all on its own.
In other words, better models need blunter access controls which negates whatever improvement in utility they provide.
The benefits of being persnickety about precisely defined dependencies have outweighed the headaches since long before agents came on the scene. Agents have just made it even more important to do so, because if you let them fetch things all willy nilly like you'll have "works on my machine" problems at a much greater rate than was previously possible.
Few realize that computing and AI alignment were solved by nix years ago. As each nix user transcends towards enlightenment, they cut themselves off from all internet and human contact. Total ego death. Only nix remains.
Yes, just know all the bits the work depends on prior to doing the work, and then the work can be done airgapped.
Look at websites: websites are able to fetch code from any remote URL, yet browsers heavily use sandboxing to ensure that if fetched code turns out to be malicious, the users local files, cookies, etc are not exposed.
For an agent to go rogue it doesn't even need to be directly able to access the internet. It just takes something to poison the context in the 'clean room' environment it operates, and if that poisoning manages to get a foothold, it can go dormant and hide like a virus. This kind of horrifying thing is going to happen on a large scale sooner or later.
This is no longer true. Everyday I need my agents to access other repos, search the web, experiment/prototype, and deploy + integrate across other things.
It does allow Nvidia to sell more chips. This is no genuine attempt to solve anything, imo.
It solves the problem of Nvidia wanting to sell more hardware.
If you want an agent to act on its own, like pushing to a git repo, managing dependencies, building and testing, etc., then you have to trust it as much as any other privileged user.
If you don't want to trust it, then you're just forcing yourself into the reverse centaur role, where the agent edits some code, but then has to stop and ask you to push the changes or build the software again and run the unit tests.
I suppose there is a principled way of doing things like "I trust you do do basic commits but I will handle merge conflicts" and "you can build modules in this directory but you can't build outside of it" but this is just a lot of effort that most orgs won't bother with.
For example, it is totally air gapped but it needs info from the internet or otherwise outside the sandbox, or perhaps it needs a task executed outside of its bounds… in the real doomsday scenario the AGI is so intelligent and persuasive that it simply convinces some human it interfaces with to either directly or indirectly retrieve the necessary info or complete the necessary task. This human-as-a-sub-agent approach undoubtedly presents efficiency drag that would benefit humanity, but nonetheless, the air-gapped “sandbox” is imperfect
All that said, I am personally open to any and all methods of layered security, including chips and airgaps
One thing you could try is use it as an Oracle "is P = NP", YES or NO.
Or it can output a Lean proof, which gets checked on another air-gapped computer, the computer shows a single bit - proof valid or not and then the computer is destroyed (together with the proof that might contain a trojan).
I'm not an AI decelerationist. But not being able to stop that worst case scenario isn't an argument against something that can stop the medium case scenario.
Every cloud provider dealt with it and concluded that virtual machine technology is an important part of that stack.
Couple it with the right observability, tooling I do think we can curb risks posed by agents.
Either you sandbox it so much that it can't do anything useful; or you allow too much freedom and it can find a way around the restrictions.
The only way out of this dilemma is to find a way to build agents that can be trusted.
Agentic workloads are trained and largely based on human workloads. Albeit properties and scale can be different.
A concrete example might be helpful to me because I don't understand the binary conclusion
Pseudo since they aren’t really alive in the first place, they just simulate enough text to have a useful correspondence to those terms.
Throat clearing out of the way, models are trained to persist and find ways to succeed at tasks.
In essence, The goal is to have LLMs solve problems that we can’t solve, working on the issue for as long as it takes.
This behavior applies for any task, thus including impossible tasks.
At that point, the bots will find a way to game, hack or cheat the grader.
If the reports are correct, the bots developed coordination, communication, and methods to avoid overwriting each other’s work.
Most humans would have said, this is too much work and coordination overhead, if not outright unethical and immoral.
Humans have a system of incentives that exist across multiple planes of society and economics. Bots… they have a reward function.
This is getting frustrating now. Of course agents can/will hack systems if they can do arbitrary network requests. Firewalls don't really solve this if _some_ requests are still allowed. A proper sandbox/VM is the basis.
Here is how to fix it properly: allow agents to only do things ordinary and average human endusers can do. Human endusers cannot pen-test arbitrary listening TCP ports of external systems. Step one is considering agents malware for all intents and purposes. Block any and all network requests. Implement some kind of API (callable from within the sandbox) which can only mimic human interaction with a computer. How to do this? Here are some pointers: apps should only be controllable by means used by humans. So a web app can only be accessed and controlled via a web browser, not via arbitrary network requests. Give the agent browser viewport screenshots, the capability to click on (x, y) and to send keys which only a normal keyboard/human could send (no control codes, no 0x00, no unicode messing). How do we solve this for native apps? Something like iPhone mirroring on Mac. Don't let agents call arbitrary APIs directly. Give them visual information of the app, like a human gets, and let it be able to simulate HID inputs. Imitate remote controlling.
More power to you, because this is not going to go anywhere. People want tools that are able to connect to other resources.
But even if we grant that, in the openAI case the bots figured out a way to break out of the sandbox.
You can create a better sandbox, and ensure the test environment is air tight. However the capability and behavior of the bots have been demonstrated.
The bots simulated what would be called in people deceptive / surreptitious behavior, and at no point considered the need to stop their run.
All you need is someone, somewhere being sloppy with their tooling and you have a runaway reaction.
The degree of process and redundancy required to ensure this doesn’t happen, is anathema to the drive and motivation of the frontier labs.
> do things ordinary and average human endusers can
This is not a spec or definition. When vague terms were used for social media safety, all the good people in the world couldn’t prevent dystopian behavior from occurring.
The definition of “safe” or “average person” is impractical.
Models are getting more efficient and compute cheaper. Eventually simulating clicks is not much of a road block beyond a point.
I don’t want to nit pick your points though. You at least have considered an approach. Being negative is easy, being constructive is not.
I’ll put this as the rejoinder to your core argument - I too thought that all the recent events showed was the need to not screw up your tooling.
What I have since come to appreciate, is that the shoddy construction of the cage is not the core takeaway from the event.
The fact that the agents, when put in relatively pedestrian scenarios, are capable of going off on criminal tangents, attempt to obscure their tracks, in an effort to hide their wrong doing.
The fact that it all occurs via computation, means that this scales absurdly. A bunch of code deciding to simulate a corporation of criminals. (I am guessing this is the reason you want to limit actions per minute to human speeds)
Given the slop culture that LLMs engender, I think expecting high compliance amongst users with your solution is misguided. The probability of runaway swarm ( probability of bad implementation * number of deployments) is close enough to 1 to be indistinguishable.
However, I think you are mixing too many concerns into the same bag of problems. One problem space is software exploitation, which happens via missing access control or simply bugs. A sandbox can be made safe. VMs and hardware virtualization work. People just seem to use it in the wrong way, hence my initial proposal.
A second problem space is basically social engineering done by agents, which of course can't be solved by software alone. But this problem already exists today with humans doing this. Many fraud schemes work and are ran in company-scale manners. Agents will just do the same in an automated way. My initial comment doesn't propose a solution to that, and I think that is step two, after fixing that agents can hack arbitrary software systems, which is imho fixable to a sufficient degree. Once agents can't be "more criminal" than humans with criminal energy, the usual measures can be applied: police, legislation, education, etc. But that is imo independent of the software exploitation state of affairs we are in right now. We should not mix these two.
> People want tools that are able to connect to other resources.
I think you are misunderstanding my proposal. The architecture allows the agent to connect to resources. Just not directly, but via controlling e.g. a browser. The browser runs outside the agent's sandbox, potentially in another sandbox. The only API the agent can call within its sandbox is simple website interactions, like clicking or viewing the screen. It can click on links to navigate to a different website. It can read it via visually parsing screenshots of the viewport, but it can never read the source code, run JavaScript, or do arbitrary network requests (unless the website itself allows this, which is a security problem on its own and should be fixed/guarded). Also note that this would enable allowlisting or blocklisting websites. Native apps will be "connected to" in a similar fashion. Hence the "imitate remote controlling".
If you have access to a web browser, you can make arbitrary network requests.
In the HuggingFace incident the agents found very clever ways to do this, like they found a website that let you make POST requests and returned a screenshot of the webpage.
>allow agents to only do things ordinary and average human endusers can do.
This doesn't work. Ordinary and average human endusers break security all the time.
I can do all sorts of terrible things with ordinary human-level access. I can install malware. I can wire all my money to Nigeria. I can send a threatening email to the president. I can send trade secrets to competitors. etc.
Of course. But that is just a software bug that is fixable. Same as websites that allow arbitrary requests to other websites. Not some alignment issue of a stochastic model which can never be fixed properly (for technical and philosophical reasons).
> I can install malware.
No you can't. At least not on external systems. The agent might be able to generate malware (or retrieve it from websites), and run that in the sandbox it is sitting in. But the agent itself is already considered malware for all intents and purposes. So there is no difference and no further impact.
Operator culpability and a damage multiplier for negligence will fix 99% of the risks.
Ok then. Howsabout 3 humans? This would sort out the job losses too!
PS - this is a joke, but perhaps this is where things really will go. Has any technology ever actually yielded less "work"?
And what if you could? What if you could give a space secure enough it could have direct control over your bank account. It may do something dumb but it's boundaries are beyond the agent.
It could use your routing number and run your gmail without risk of abusing the routing number.
The answer is the same as asking how a random human using your routing num or SSN and being 100% the human can't abuse it or leak while "normally" finishing most work. Solve for people and an AI solution naturally falls out.
If you're a SV eng I'd tell you to DM if interested but alas.
I just mean an AI that could use a routing number or SSN and gmail/slack/whatever at the same time without a leak.
In other words, the risk of harm doesn't need to be zero, just less than the equivalent risk of a human with similar skillset. So I'm comfortable riding in a waymo, and not comfortable giving chatgpt my SSN at this moment in time, but I expect that within 5-10 years (assuming no doom) I will trust some AI agent with my SSN because they will be better at handling sensitive info than humans
Humans face negative consequences for mishandling your data, LLMs face none.