1,229 karma · joined September 23, 2016
It’s underappreciated that we would already be in a pretty absurdly wild tech trajectory just due to compute hyperabundance even without AI.
In most cases I would never have undertaken those projects at all without AI. One of the projects that is currently live and making me money took about 1 working day with Claude Code. It’s not something I ever would have started without Claude Code, because I know I wouldn’t have the time for it. I have built websites of similar complexity in the past, and since they were free-time type endeavors, they never quite crossed the finish line into commerciality even after several years of on-again-off-again work. So how do you account that with a time multiplier? 100x? Infinite speedup? The counterfactual is a world where the product doesn’t exist at all.
This is where most of the “speedup” happens. It’s more a speedup in overall effectiveness than raw “coding speed.” Another example is a web API for which I was able to very quickly release comprehensive client side SDKs in multiple languages. This is exactly the kind of deterministic boilerplate work LLMs are ideal for, and that would take a human a lot of typing, and looking up details for unfamiliar languages. How long would it have taken me to write SDKs in all those languages by hand? I don’t really know, I simply wouldn’t have done it, I would have just done one SDK in Python and said good enough.
If you really twist my arm and ask me to estimate the speedup on some task that I would have done either way, then yeah I still think a 100x speedup is the right order of magnitude, if we’re talking about Claude Code with Opus 4.1 specifically. In the past I spent about a five years very carefully building a suite of tools for managing my simulation work and serving as a pre/post-processor. Obviously this wasn’t full-time work on the code itself, but the development progressed across that timeframe. I recently threw all that out and replaced it with stuff I rebuilt in about a week with AI. In this case I was leveraging a lot of the learnings I gleaned from the first time I built it, so it’s not a fair one-to-one comparison, but you’re really never going to see a pure natural experiment for this sort of thing.
I think most people are in a professional position where they are sort of externally rate limited. They can’t imaging being 100x more effective. There would be no point to it. In many cases they already sit around doing nothing all day, because they are waiting for other people or processes. I’m lucky to not be in such a position. There’s always somewhere I can apply energy and see results, and so AI acts as an increasingly dramatic multiplier. This is a subtle but crucial point: if you never try to use AI in a way that would even hypothetically result in a big productivity multiplier (doing things you wouldn’t have otherwise done, doing a much more thorough job on the things you need to do, and trying to intentionally speed up your work on core tasks) then you can’t possibly know what the speedup factor is. People end up sounding like a medieval peasant suddenly getting access to a motorcycle and complaining that it doesn’t get them to the market faster, and then you find out that they never actually ride it.
I wonder, have you sat down and tried to vibecode something with Claude Code? If so, what kind of multiplier would you find plausible?
My one concrete pushback to the article is that it states the inevitable end result of vibe coding is a messy unmaintainable codebase. This is empirically not true. At this point I have many vibecoded projects that are quite complex but work perfectly. Most of these are for my private use but two of them serve in a live production context. It goes without saying that not only do these projects work, but they were accomplished 100x faster than I could have done by hand.
Do I also have vibecoded projects that went of the rails? Of course. I had to build those to learn where the edges of the model’s capabilities are, and what its failure modes are, so I can compensate. Vibecoding a good codebase is a skill. I know how to vibecode a good, maintainable codebase. Perhaps this violates your definition of vibecoding; my definition is that I almost never need to actually look at the code. I am just serving as a very hands-on manager. (Though I can look at the code if I need to - have 20 years of coding experience. But if I find that I need to look at the code, something has already gone badly wrong.)
Relevant anecdote: A couple of years ago I had a friend who was incredibly skilled at getting image models to do things that serious people asserted image models definitely couldn’t do at the time. At that time there were no image models that could get consistent text to appear in the image, but my friend could always get exactly the text you wanted. His prompts were themselves incredible works of art and engineering, directly grabbing hold of the fundamental control knobs of the model that most users are fumbling at.
Here’s the thing: any one of us can now make an image that is better than anything he was making at the time. Better compositionality, better understanding of intent, better text accuracy. We do this out of the box and without any attention paid to promoting voodoo at all. The models simply got that much better.
In a year or two, my carefully cultivated expertise around vibecoding will be irrelevant. You will get results like mine by just telling the model what you want. I assert this with high confidence. This is not disappointing to me, because I will be taking full advantage of the bleeding edge of capabilities throughout that period of time. Much like my friend, I don’t want to be good at managing AIs, I want to realize my vision.
Whenever I experiment with agent frameworks that spawn subagents with scoped subtasks and restricted context, things go off the rails very quickly. A subagent with reduced context makes poorer choices and hallucinates assumptions about the greater codebase, and very often lacks a basic sense the point of the work. This lack of situational awareness is where you are most likely to encounter js scripts suddenly appearing in your Python repo.
I don’t know if there is a “fix” for this or if I even want one. Perhaps the solution, in the limit, actually will be to just make the big-smart models faster and faster, so they can chew on the biggest and most comprehensive context possible, and use those exclusively.
eta: The big models have gotten better and better at longer-running tasks because they are less likely to make a stupid mistake that derails the work at any given moment. More nines of reliability, etc. By introducing dumber models into this workflow, and restricting the context that you feed to the big models, you are pushing things back in the wrong direction.
Then after using the new model for a few months you get used to it, you feel like you know what it should be able to do, and when it can’t do that, you’re annoyed. You feel like it got worse. But what happened is your expectations crept up. You’re now constantly riding it at 95% of its capabilities and hitting more edge cases where it messes up. You think you’re doing everything consistently, but you’re not, you’ve dramatically dialed up your expectations and demands relative to what you were doing months ago. I don’t mean “you,” I mean the royal “you”, this is what we all do. If you think your expectations haven’t risen, go back and look at your commits from six months ago and tell me I’m wrong.
Last night I set up my 11 year old son with Claude 4, with MCP enabled for filesystem modification, reasoning that the LLMs are finally at the level of capability where they can reliably just do things. And I was right - Claude put together a browser JS game according to his description in seconds, and iterated on it several times to incorporate his suggestions.
But then it hit the limit of how big of a file it can comfortably write out in one go, started making incomplete edits, and basically fell into a snarl of endlessly trying and failing to write files due to file length limitations. I had to step in and tell Claude to split the code up into several files, something my son wouldn’t have known to do.
If I hadn’t been there to tell Claude how to work around its own UI and tool limitations, it would have likely blown through the rest of its context window and totally failed at the task. I imagine this is a common experience for people.
Most people wouldn’t know how to set up Claude with MCP in the first place. A surprising number of people who seem relatively aware of LLM technology aren’t aware of MCP, especially if their workflow is centered on IDE integration. To be fair, these are probably the people who need MCP the least, but MCP (and, in general, agentic tool use) is definitely closer to how normal people will get value out of LLMs.
It may seem stupid and trivial, but telling Claude to directly edit or debug a file that exists on your hard drive is actually a multiples-faster and smoother experience than doing the laborious copy-paste exercise many people are still engaging with. This is just as true for code as it is for writing documents.
The LLM companies seem to be quite aware of this, hence products like Claude Code, the Claude computer use suite, all the Gemini android screen/camera share integration, and GPT Codex. It’s a rocky process but at some point soon we will cross the threshold where it just works. But right now, it doesn’t just work, and so it’s not really that much faster or more efficient than doing a task yourself, especially if you aren’t intimately familiar with all the quirks and limitations of LLMs.
Unlike traditional methods that rely heavily on scalarization, heuristic tuning, or exhaustive searches, globalMOO leverages a proprietary agent-based system to achieve highly efficient optimization across diverse and conflicting objectives - converging on optima 1000x faster than traditional methods in benchmarked high-dimensional use cases.
Key highlights:
True Multi-Objective Optimization: No scalarization or subjective weighting needed. Directly solves problems with objectives in differing units and scales, simultaneously.
Inverse Solution Capability: Model-agnostic and adept at optimizing black-box, physics-based, or AI-driven models. No need to expose or to modify the contents of your algorithms.
Data-Efficient: Significantly fewer model evaluations compared to standard methods (MOEAD, DNSGA2, NSGA2), often requiring orders of magnitude fewer iterations.
High Scalability: Easily scales to 200+ input variables and hundreds of objectives.
Versatile Integration: Provides robust SDKs in Python, C#, PHP and JavaScript for interacting with the Web API, and various local DLL/.so interfaces for local installation, facilitating seamless integration into existing workflows.
We've applied globalMOO in diverse fields like petroleum engineering, manufacturing, and shipping, consistently outperforming existing algorithms in terms of computational efficiency and convergence speed.
Register for a free trial of the web API at https://app.globalmoo.com/ or check out a variety of example implementations to get you up and running at https://github.com/globalMOO.
If your model of reality makes good predictions and mine makes bad ones, and I want a more accurate model of reality, then I really shouldn’t just make small provisional and incremental concessions gerrymandered around whatever the latest piece of evidence is. After a few repeated instances, I should probably just say “oops, looks like my model is wrong” and adopt yours.
This seems to be a chronic problem with AI skeptics of various sorts. They clearly tell us that their grand model indicates that such-and-such a quality is absolutely required for AI to achieve some particular thing. Then LLMs achieve that thing without having that quality. Then they say something vague about how maybe LLMs have that quality after all, somehow. (They are always shockingly incurious about explaining this part. You would think this would be important to them to understand, as they tend to call themselves “scientists”.)
They never take the step of admitting that maybe they’re completely wrong about intelligence, or that they’re completely wrong about LLMs.
Here’s one way of looking at it: if they had really changed their mind, then they would stop being consistently wrong.
I think LLMs have uncovered what we have always known in this industry: that people are, by default, bad at communicating their intent clearly and unambiguously.
If you express your intent to an LLM with sufficient clarity and disambiguation, it will rarely screw up. Often, we don’t have time to do this, and instead we aim for the sweet spot of sufficient but not exhaustive clarity. This can be fine if you are experienced with that particular LLM and you have a good feel for where its sweet spot actually is. If you miss that target, though, the LLM will not correctly infer your intended subtext. This is one of the things that requires experience. In fact, even the “same” LLM will change in its behavior and capabilities as it undergoes fine tuning. Sometimes it will even get worse at certain things.
All of this is to say, of course, you’re right that it’s not a compiler. But I think people fail in their application of LLMs for much the same reason that novice coders fail to get compilers to guess what they intended.
I’m currently using them to port a client-side API SDK into multiple languages. This would be a pain in the ass time consuming task but is a breeze with LLMs because the exact behavior I want is clearly defined and relatively deterministic, and it’s also straightforward to test that I’m getting what I intend. The LLM thus gets done in 3 days what would take me 3 weeks (or more) to do by hand.
If the complaint is that it can’t do X, where X is something that would clearly require full AGI and likely true superintelligence — in this case expecting instantaneous, correct code that solves novel problems on the first try - then I have to insist that people are actually expecting Claude to be a Culture Ship Mind, implicitly. They just don’t realize that what they’re asking for his hard, which is itself a psychologically interesting fact, I suppose.
We get it. They’re not superintelligent at everything yet. They couldn’t infer what you must’ve really meant in your heart from your initial unskillful prompt. They couldn’t foresee every possible bug and edge case from the first moment of conceptualizing the design, a flaw which I’m sure you don’t have.
The thing that pushes me over the line into ranting territory is that computer programmers, of all people, should know that computers do what you tell them to.
If what I’m requesting an improvement to an existing code, I paste the whole code if practical, or if not, as much of the code as possible, as context before making request for additional functionality.
Often these days I add something like “preserve all currently existing functionality.” Weirdly, as the models have gotten smarter, they have also gotten more prone to delete stuff they view as unnecessary to the task at hand.
If what I’m doing is complex (a subjective judgement) I ask it to lay out a plan for the intended code before starting, giving me a chance to give it a thumbs up or clarify its understanding of what I’m asking for if it’s plan is off base.
So there are solutions to hallucination, but the product has to prioritize that to the extent that it reduces the model’s effectiveness as a general tool.
• 7-10 GPT-4 level models
He only gets to claim this is true if you count a bunch of different versions of the same base model, or if you’re willing to say that some models that outperform GPT-4 on some benchmarks count as being GPT-4 level. I don’t think Marcus was right in spirit, here.
• No massive advance (no GPT-5, or disappointing GPT-5)
Seems too unquantifiable to judge, I would call o1 a massive advance over 4o, but I’m sure Marcus would not.
• Price wars
I guess so? From what I’ve read, the frontier models companies are still profitable, and OpenAI now has a $200/mo commercial model, hardly the action of a company deciding its prices purely to undercut the competition.
• Very little moat for anyone
It still seems like the only companies who have pulled off frontier model capabilities have spent many millions of dollars doing it. I think this might become true next year but I don’t think this can be judged as correct based on what we saw in 2024 alone.
• No robust solution to hallucinations
You only use words like “robust” in a prediction like this so you have room to weasel out of it later when the hallucinations diminish greatly but don’t quite go extinct.
• Modest lasting corporate adoption
My industry is oil and gas. A pretty hidebound and conservative industry. Adoption of LLMs has been massive.
• Modest profits, split 7-10 ways
Define modest.
I score Marcus at 0/7, at best 2/7.
Much better to just stay in the market, knowing that there will be crashes and you will have days where the numbers look awful, because they’ll look great again in a few years.