3,824 karma · joined March 10, 2017
It sounds remarkable, and I didn't believe it at first. But after more reading, I am convinced an influential part of Anthropic leadership really is something like a religious movement
(I don't have a theology degree, but I do have an English degree. I have read parts of KJV, and it has literary significance, but it is not attempting to be a bare literal translation)
Sin: A Slop Colon!
People ask AI to do things either they haven't done, cannot do, or don't want to.
So when the AI produces something that basically works and looks like a plausibly good attempt, they jump to the conclusion that AI must be excellent at that task.
The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.
The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.
The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.
This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
We were not building the feature because we wanted it. We were building it because we thought other people expected it.
Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.
Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.
Where does the market's "action" come from? What is the driver?
Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.
This left me lost.
Nobody in this ecosystem, I thought, is actually in control here.
Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.
So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.
It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.
Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.
You can have billions in nominal units but it is inflated to worthlessness when resources or labour disappear
It's that they're a very insular, incestuous sub-culture whose origins come from early 2000s science fiction message boards and which has been distilled through the cod utilitarianism of the Effective Altruism people.
That doesn't mean the ideas are "bad" because "bad" is some infectious transitive property. But it does mean they are intellectually homogeneous; disdain experts in other fields; treat ethics as mathematical functions with infinite-sized terms; and whose opinions about AI pre-date the development of LLMs themselves.
They also - I will be direct here - have a history of handwaving bad behaviour, conflicts of interest, and recklessness.
When cliques take over something - businesses, technologies, politics - it means intellectual diversity leaves the room and important decisions get made, not due to practical reasons, but increasingly arcane wranglings within the clique itself
It has definitely accelerated my learning: through the process of writing rigorous plans, I have learned so much about network programming... the intricacies of HTTP2, how to design a fast MITM proxy, performance pitfalls in SSL...
These are things I studiously avoided learning for a decade, now demystified, mostly through Claude patiently rendering many figures and diagrams.
And with Fable, I was able to ship a working Envoy Rust extension in a one-shot.
On the other hand, my grasp on the details is tenuous. Every so often the agent can't progress and I am stuck for ideas, hitting different prompts like they were slot machines. I forget how to solve problems by myself.
I can feel myself becoming supine and dependent on something I know is going to become more expensive, a dependency AI vendors will seek to exploit. It has happened quickly. I do not really consent; it is forced on me by my employer.
I used to have several good answers, but now I use Claude for everything, I can't remember any
You can't make that assertion when these same people made the LLM revolution happen, by securing the capital and human resources to realise their dream / nightmare
The hedge funds are trying to back out. I think they realise the web of financing makes them double (triple? quintuple?) exposed.
Amodei wants to manage the narrative and make the slowdown look at least semi-deliberate.
The cynic in me would argue the safety and alignment play has always filled a role of plausible deniability when plans go off track. "Oh well we had to pull the model", etc
What I would ask is this: why should I use your software when I could generate my own?
For a relatively low cost in tokens, I could use Cursor & Fable 5.1 to generate a decent text editor of my own, if I constrained it to use high quality dependencies
So this is kind of the real quandary of OSS development: is there really any point creating tools for developers any more?
My very first experience of RN was having to update an app to handle the mandatory change to 64 bit APKs
Which required a massive update not only to several dependencies but also xcode, iOS frameworks, the weird objective C caches that get smuggled into node_modules, _as well as_ all the rigmarole or making the 64 bit APK build work
It was completely miserable
It's been on my mind a lot, how I've lost touch with all this
But basically I don't think those problems are all meant to be solved
Yes, the supernatural events described on stage are very difficult to pull off even with modern effects. But they were impossible with Jacobean setups. And that's kind of the point
There is always a tension between what is seen and spoken in that play. We're meant to get the sense that Macbeth is partly talking himself into his frenzy
Supernatural events happen but they kind of have a bathetic, eye-rolling quality to them, like the prophecy that "Great Birnam Wood to high Dunsinane Hill / Shall come against him"
Which YEAH, it does, in that Macbeth's enemies construct a bunch of battering rams from the forest timber. You are supposed to groan at that. Magic is way weaker than any of the human beings and their ordinary vices
I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.
(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)
For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.
If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.
This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.
I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily
I can't remember the exact measurement but he was showing a set of curves that were simply too neat for a relatively small sample size - roughly 100 or so plastic transistors. In reality you cannot get that neat an effect without a huge sample population
This is a thought I want to pull on
I have a view that organisations, with their network of misaligned incentives, mean that companies act something like an alien lifeform. Something with a decentralised brain, unable to plan beyond immediate stimuli
Companies consume and self-cannibalise. They are both aggressively rational in pursuit of profit but irrational when it comes to preserving their ecosystems
Many companies look like groups of people where nobody has total control. Everyone is reacting to some stakeholder, even the CEO is terrified of "the shareholders". So these organisations cut off healthy limbs (layoffs), engage in lurid rituals (AI), and wastefully destroy the natural resources of the humans who work for them
I first heard this idea in a James Bridle lecture, where he suggested that the strange behaviour of companies was a good starting point for understanding what an alien lifeform might behave like. And he observed the ways that humans and machines now mirror each other.
Inside our society, controlled by these aliens, humans act like machines (Taylorism at work, self-censorship for algorithms at play). Machines act like humans (AI, spam, computer programs mediating relationships).
Underneath it is a frantic need to quantize everything so it can be dominated