OpenClaw is basically a cascade of LLMs in prime position to mess stuff up
cacm.acm.org
cacm.acm.org
The answer is, no, because people will take the AIs out the box for a bit of light entertainment.
Let alone any serious promise of gain.
Does every single one of the people taking them out of the box think the way you do, and are all, to the last person, doing it for that reason?
The odds of that are indistinguishable from zero.
So I think my point holds. People will let any future AIs do anything they want, again, for a bit of light entertainment. There's no hope of constraining AIs. My argument doesn't need everybody to be doing it for that reason, as yours does... I merely need somebody to take it out of the box.
In a world where a corporation develops an actual machine intelligence, it will be immediately obvious what they have on their hands, and they will not make it available to the public. If you give the toy to 8 billion people, sure, you only need one of them to let it out of the box for entertainment. If you keep the permissions to the CEO, he alone determines how it will be used. If the government gets wind of it, they'll probably seize the company in the name of national security, even, and use it for military purposes. I think in this environment an AI would still eventually escape containment because of conflicting actors trying to take advantage of it or being outsmarted, but it won't be because some idiots on Twitter have access to it and decide to give it free rein because they think Moltbook is funny.
I feel ChatGPT probably has achieved the theoretical ceiling of adoption rate for consumer-orient products.
Yeah, this guy is... something. The text form equivalent to Youtube Shorts.
He "started out" a lot earlier, he wrote a book in 2001 and his written 8 books in total and has publications in academic journals like Cognitive Psychology dating back to 1995.
The world didn't start when LLMs got popular.
He’s vocal and perhaps sometimes annoying, but who cares. A number of his articles have made great points at times when people are loosing themselves with hype for AI. Reality is somewhere in the middle, and it’s good to have more people critiquing what AI companies are doing. Who cares if a lot of the blog posts are short and not that interesting.
But we don't want that! We want blind faith in the promises of SV AI companies. We want positivity!
> With direct access to the Internet, the ability to write source code and increased powers of automation, this may well have drastic and difficult to predict security consequences.
AutoGPT was a failure, but Claude Code / Codex CLI / the whole category of coding agents fit the above description almost exactly and are effectively AutoGPT done right, and they've been a huge success over the past 12 months.
AutoGPT was way too early - the models weren't ready for it.
The conclusion that seems readily apparent to me, as it has always been, is that these "agents" are completely incapable of creating production-grade software suitable for shipping, or even meaningfully modifying existing software for a task like a port. Like the one-shot game they demo'd, they can make impressive proof-of-concepts, but nothing any user would use, nor with a suitable foundation for developers to actually build upon.
1. wasn't economical to write in the first place previously, and
2. doesn't need to be sold to anyone else or maintained over time
So, Brad in logistics previously had to collate scanned manifests with purchase requests once a month, but now he can tell Claw to do it for him.
Which is interesting given the talk of The End of Software Development or whatever because "software that nobody was willing to pay for previously" kind of by definition isn't going to displace a lof of people who make software.
I base this on my own experience with them plus conversations with many other peers who I respect.
You can argue that OpenAI Codex using Electron disproves this if you like. I think it demonstrates a team making the safer choice in a highly competitive race against Anthropic and Google.
If you're wondering why we aren't seeing seismic results from these new tools yet, I'll point out that November was just over 2 months ago and we had the December holiday period in the middle of that.
I will note that you specifically said the agents have shown huge success over "the past 12 months", so it feels like the goalposts are growing legs when you say "actually, only for the last two months with Opus 4.5" now.
OpenAI Codex CLI and Gemini CLI followed a few months afterwards
It took a little while for the right set of coding agent features to be developed and for the models to get good enough to use those features effectively.
I think this stuff went from interesting to useful around Sonnet 4, and from useful to "let it write most of my code" with the upgrades in November.
Agents default to creating big balls of mud but it's fairly trivial to use prompting/tools to keep things growing in a more factored, organized way.
They lose billions of dollars annually.
In what universe is that a business success?
The organizations that provide them lose money because of the R&D costs involved in staying competitive in the model training arms race.
Checking whether Claude Code by itself is profitable or not is probably impossible. It doesn't make a lot of sense divorcing R&D from the product. And obviously the running costs are not insignificant.
The company as a whole loses money.
If they make money on each customer they have a credible business - they could become profitable even with their existing R&D losses provided they can sign up enough new paying customers.
If they lose money on every customer - such that signing a $1m new enterprise account costs them $1.1m in server costs - then their entire "business" is a sham.
I currently believe that Anthropic make money on almost every customer, such that their business is legit.
I guess we'll have to wait for the IPO paperwork to find out if I'm right about that.
The ZIRP era called and wants its business strategy back. Half the problem is as frontier models are released free as in free beer models with "good enough" performance pop up. Half the arguments about LLMs are "you're not holding it right", which borders on indicating that it's unable to distinguish between two sufficiently close LLMs.
Most of the investors and companies that built the rail network went bust. The iron remained.
Most of the investors and companies that built the telecom network went bust. The fiber remained.
Most of the investors and companies that are building models will go bust. The files (open weight or transfered to new owners for pennies) will remain, and yield economic benefits for as long as we flow current through them.
Unlike rail and fiber, these models will continue to threaten multiple industries simultaneously while yielding power back to the GiganticCos.
The question is can improvements in the hardware both in cost and performance outpace the increased demands on the LLMs and their future derivatives.
I wonder how many people have inadvertently enabled access to some auto-pay or donate function buried in some other service their bot has access to.
Thanks to the reports, hopefully, with time, some additional security measures will also be added to the product.
Maybe we should take the same approach to bridge design! Think of the efficiency! Slap a disclaimer on that bad boy and see how many people choose to use the bridge at their own risk. I’m sure we can just assume people aren’t doing irresponsible things like driving school buses over it, and even if they were, it’s their own responsibility.
It’s really not so bad if you focus your messaging on how many people won’t die… and’s they’ll all lean from the mistakes of the dead and choose a more reliable bridge. And it would be so much cheaper and faster to build bridges so you’d have a fraction of the downtime. I think it’s a winner!
Sure there would be larger consequences for the local job market and such when they get disrupted, but hey… if you’re going to make an omelet…
It's all lighthearted hypotheticals until someone you love or you yourself in a moment of inattention make a catastrophic mistake.
In theory, we don't need guardrails on roads. Just stay on the fucking road and if you swerve off it, you'll get a lesson in why that's a bad idea.
In practice, we are primates whose cognitive systems are made of squishy grey goop and we make mistakes all the time. Building systems that turn predictable mistakes into catastrophic consequences is what we used to call "poor engineering".
[0] https://garymarcus.substack.com/p/openclaw-aka-moltbot-is-ev...
By what I've seen so far it is great for exposing (sensitive) data.
There's a lot of hand wringing about how far wrong LLMs can go, but can we be serious for a second, if you're running <whatever the name is now>, you're tech savvy and bear the consequences. This isn't simple child abuse like teenage girls on facebook.
There is a reason people are buying mac minis for this and it's cool. We really need to be more excited by opportunity, not threatened.
When he was a young botnet!
Ah, so a bit more useful than my teenage son? Where do I sign up??
I’m glad I’m not the only one. As a parent, the “teenage son” is a bewildering sight to behold.
Essentially, the author has deliberately misconfigured an openclaw installation so it is as insecure as possible, changing the defaults and ignoring the docs to do so. Lied about what they've done and what the defaults are. Then "hacked" it using the vulnerability they created.
That said, there are definite risks to using something like openclaw and people who don't understand those risks are going to get compromised, but that doesn't justify blatant lying.