Famously, some of those subpar products are now household names who were able to stake out their place in the market because of their ability to move quickly and iterate. Had they prioritized long-maintainable code quality rather than user journey, it's possible they wouldn't be where they are today.
"Move fast and break things" wasn't a joke; Mark really did encourage people to build faster and it helped cement their positioning. Think of the quantity of features FB shoveled out between 2009-2014 or so, that just wouldn't have been possible if their primary objective was flawless code.
The code isn't the product, the product is the product. In all my years of engineering I've yet to have an end-user tell me they liked my coding style, they've always been more concerned with what I'd built them.
Famously gmail was very good quality web app and code(can't say the same today) surely not the product of today's "fast iteration" culture
There are lots of companies doing well producing high quality products out of the gate today though, look at Linear.
Both approaches are valid for building sustainable enduring businesses.
Sure exploiting human attention to sell them ads, it's not technological marvel, I'd say psychological
Technology wise they created React, PyTorch, GraphQL and Llama is open weights.
Lest not forget when Hotz said he could easily fix Twitter's search functionality only to give up after 3 months [1]. When immensen scale is involved things do become difficult.
Like we have a comment below taking a shot at Phillip Morris [2]. Lets see you grow, process, and distribute 1/100 the quality of cigarettes. The end product might not be that great for society but it's not trivial to do it either.
[1]: https://www.theverge.com/2022/11/22/23472869/george-hotz-twi...
I don't really see it as an example of what you're saying so much as an example of success as a product having to do with far more than explicit product features. You can see a similar dynamic in other natural monopoly markets. The NBA didn't necessarily do anything particularly right product wise, for instance. They just got the best players because basketball was historically more popular in the US than other countries, and the ABA made some stupid decisions that let the NBA win out in the US.
Hell, the US itself didn't do a whole lot "right" aside from not being in Europe when Europe decided to destroy itself, being better-positioned than other potential competitors like Canada, Mexico, and Australia simply because North America is best positioned to trade with both Europe and Asia and the US is more temperate than Canada or Mexico. But we sure like to tell ourselves stories about everything we did right.
Yes, both impact each other, and if facebook was just shoddily coupled together it probably wouldn't be a great product, but so many engineers get into this mindset of needing finely tuned clean code vs a working product.
When people were switching, how many decided to use facebook because the coding style of the backend was really clean and had good isomorphism?
There's a balance with this, but so much of it's success was being at the right place at the right time, and if you spend a massive amunt of time not building product and just building good code, you're going to hit a failure.
Some of us are building prototypes, and others are building software with a 10 year horizon or work with sensitive personal data.
So on the one side you get people that are super efficient cranking things out, and others that read this and feel appalled anyone would be comfortable with this and see software engineering as theory building. Neither are really wrong here, but the context / risk of what people are working on always seems to be missing when people talk about this.
I am just as pissed as anybody else at the lack of certainty in the future.... Thought I had my career made. So it's not that I don't emphatize with engineers in my shoes.
* There is no objective reality, there isn't one in physics, it's just a non argument * "LLM's help a damn lot in speed of development" That may be your experience and my whole point was arguing that speed may not matter * "Genie is out of the bag, you have to go with the flow, adapt or else" I choose else if this the apex of the argument
Here is a fun think to try: demonstrate that you and me are seeing the same color in the same way? You might find some papers trying to prove it but you will se they are all based on subjective answers of people.
So while at macro level the reality seems hard and objective at the micro level it is not.
And ofc at macro level we live in a very objective reality. This is the basis of science.
The same reason we've always needed them:
1: They prevent regressions. (IE, bugs in features that were shipped and already working.)
2: They are very easy to run at the push of a button in your IDE. (But in this context the LLM runs them.)
3: They run in CI. This is an important line of defense in making sure a pull request doesn't introduce a bug.
Now, depending on what you're writing, you might not need unit tests! Perhaps you're trying to get a minimum viable product out the door? Perhaps you're trying to demo a feature to see if it's worth building? Perhaps you're writing a 1-off tool that you'll run a few times and throw away?
But, understand that if you're writing an industrial-strength program, your unit tests help you ship bug-free software. They allow you to do some rather major refactors, sometimes touching areas of the codebase that you only lightly understand, without needing to manually test everything.
(And, to keep it in context,) your LLM will also have the same benefits from this tired-and-true process.
I have tools I've been using for 25 years that I still think I could be using better.
One of the more "engineering" like skills in using this stuff is methodically figuring out what's a superstition and what actually works.
> One of the more "engineering" like skills in using this stuff is methodically figuring out what's a superstition and what actually works.
The problem is there are so many variables and the system is so chaotic that this is a nearly impossible task for things that don’t have an absolutely enormous effect size.
For most things you’re testing, you need to run the experiment many many times to get any kind is statistically significant result, which rules out manual review.
And since we have tried and failed to develop objective code quality metrics, you’re left with metrics like “does this pass the automated test or not!”, but that doesn’t tell you whether the code is any good, or whether it is overfitting the test suite. Then when a new model comes out, you have to scrap your results and run your experiments all over. This is engineering of the laws of physics were constantly changing, and I lived in that universe, I think I’d take my ball and go home.
There's always been a bit of magic to being a programmer, and if you look at the cover of SICP people like to imagine that they are wizards or alchemists. But "vibe engineering" moves that to a whole new level. You're a wizard mixing up gunpowder and sacrificing chickens to fire spirits before you light it. It's not engineering because unless the models fundamentally change you'll never be able to really sort the science from the superstition. Software engineering already had too much superstition for my taste, but we're at a whole new level now.
I had Claude Code do some work which I pushed as a branch to GitHub. Then I opened a PR so I could more easily review it and added a bunch of notes and comments there.
On a hunch, I pasted the URL to that PR into Claude Code and said "use the GitHub API to fetch the notes on this PR"...
... and it did exactly that. It guesses the API URL, fetched the JSON and read my notes back to me.
I told it to address each note in turn and commit the result. It did.
If a future model changes such that it can no longer correctly guess the URL to fetch JSON notes for a GitHub PR I'll notice when this trick fails. For the moment it's something I get to tuck in my ever expanding list of things that Claude (and likely other good models) can do.
You asked it to do a single easily verifiable task and it did it. You don’t know whether that’s something it can do reliably until you test it sure.
An example of a possible superstitious action would be always adding commands as notes in a PR because you believe Claude gives PR notes more weight.
That’s something that sounds crazy, but it’s perfectly believable that some artifact of training could lead some model to actually behave this way. And you can imagine that someone picking up on this pattern could continue to favor writing commands as PR notes years after model changes have removed this behavior.
I wrote up another real world example of how I use Claude Code this afternoon: https://simonwillison.net/2025/Oct/8/claude-datasette-plugin...
The black box, random, chaotic nature of LLMs virtually ensures that you will pick up superstitions even if they aren’t as obvious as the above. Numbered lists work better than bullets. Prompts work better if they are concise and you remove superfluous words. You should reset your context as soon as the agent starts doing x.
All of those things may be true. They may have been true for one model, but not others. They may have never been generally true for any model, but randomness led someone to believe they were.
I've been asking it for "credible" reports on topics, because when I use that word its thinking trace seems to consider the source of the information more carefully. I've noticed it saying things like "but that's just a random blog, I should find a story from a news organization".
But... I haven't done a measured comparison, so for all I know it has the same taste in sources even if I don't nudge it with "credible" in the mix!
I've used this argument for real in the past with people who complain that it's unethical to set sightless people up with vision LLM tools because those tools are unreliable and make mistakes. My counter is that a) so are guide dogs and b) it's rude to discount the agency of people with accessibility needs in evaluating and selecting tools for themselves.
It's still OK to use dogs as an analogy. In this case the analogy is to unreliable tools, and dogs are unreliable tools.
I don't find "stochastic parrot" offensive as an analogy, even though it's got parrots in it.
If not, why not?
Is it because the parrot analogy is directly critical of LLMs, while the dog one is more of an excuse for their failings?
I’ve found it much more useful in life to always assume I’m not doing something to its full potential.
We all agree, the people that _feel_ the most productivity spike are the sub-par engineers. That shouldn't be controversial, and it's even predictable. But their volume can't be taken as an argument one way or the other.
The question is, are there _any_ good engineers that don't just feel more productive, but objectively are.
Let's even concede that speed of delivery indeed is an actual factor, there has to be a threshold. There's a point where people just don't care how fast you're putting out features, because your product has found its sweet spot and is perfectly scratching its market's itch. A few will clearly notice when the line of diminishing returns is crossed and if they can reset their outlook to fit the new context, a continuous focus on speed of delivery will look increasingly obsessive and nonsensical.
But that's the reality of the majority of the software development world. Few of us work on anything mission critical. We could produce nice sane software at a reasonable pace with decent output, but we're sold the idea that there's always more productivity to squeeze and we're told that we really really want that juice.
Trivial software is software for which
- the value of which the software solution is widely accepted and widely known in practice and
- formal verification exists and is possible to automate or
- only has a single satisfying possible implementation.
Most software is non-trivial.
There will always be:
- bugs in implementation
- missed requirements
- leaky abstractions
- incorrect features with no user or business value
- problems with integration
- problems with performance
- security problems
- complexity problems
- maintenance problems
in any non-trivial software no matter how "good" the engineer producing the code is or how "good" the code is.
These problems are surfaced and reduced to lie within acceptable operational tolerances via iterative development. It doesn't matter how formal our specifications are or how rigorous our verification procedures are if they are validated against an incorrect model of the problem we are attempting to solve with the software we write.
These problems can only be discovered through iterative acceptance testing, experimentation, and active use, maintenance, and constructive feedback on the quality of the software we write.
This means that the overall quality of any non-trivial software is dominated by the total number of quality feedback loops executed during its lifetime. The number of feedback loops during the software's lifetime are bound by the time it takes to complete a single synchchronous feedback loop. Multiple feedback loops may be executed in parallel, but Amdahl's law holds for overall delivery.
Therefore, time to delivery is the dominant factor to consider in order to produce valuable software products.
Your slower to produce, higher quality code puts a boundary on the duration of a single feedback loop iteration. The code you produce can perfectly solve the problem as you understand it within an iteration, but cannot guarantee that your understanding of the problem is not wrong. In that sense, many lower quality iterations produces better software quality as the number of iterations approaches infinity.
I'll reply just to that as it being the tldr. First of all tech debt is a thing and it's the thing that accumulates mostly thanks to fast feedback iterations. And in my experience the better the comunication, to get the implementation right, and the better the implementation and it happens that you can have solid features that you'll unlikely ever touch again, user base habit is also a thing, continuing on interating on something a user knows how to use and changing it is a bad thing. I'd also argue it's bad product/project management. But my whole original argument was why we'd need to have a greater speed in the first place, better tooling doesn't necessarily means faster output, productivity as well isn't measured as just faster output. Let me make a concrete example, if you ask an LLM X to produce a UI with some features, most of them will default to using React, why? Why can't we question the current state of web instead of continue to pile up abstractions over abstractions? Even if I ask the LLM to create a vanilla web app with HTML, why can't we have better tooling for sharing apps over the internet? The web is stagnant and instead of fixing it we're building castles over castles over it
There aren't many features that you'll never touch again. There are some, but they usually don't really reach that stage before they are retired. Things like curl, emacs, and ethernet adapters still exist and are still under active development after existing for decades. Sure, maybe the one driver for an ethernet adapter that is no longer manufactured isn't very active, but adding support for os upgrades still requires maintenance. New protocols, encryption libraries and security patches have to be added to curl. emacs has to be specially maintained for the latest OSX and windows versions. Maintenance occurs in most living features.
Tools exist to produce extra productivity. Compilers are a tool so that we don't have to write assembly. High-level interpreted languages are a tool so we don't have to write ports for every system. Tools themselves are abstractions.
Software is abstractions all the way down. Everything is a stack on everything else. Including, even, the hardware. Many are old, tried and true abstractions, but there are dozens of layers between the text editor we enter our code into and the hardware that executes it. Most of the time we accept this, unless one of the layers break. Most of the time they don't, but that is the result of decades of management and maintenance, and efforts sometimes measured in huge numbers of working hours by dozens of people.
A person can write a rudimentary web browser. A person cannot write chrome with all its features today. The effort to do so would be too great to finish. In addition, if finished, it would provide little value to the market, because the original chrome would still exist and have gained new features and maintenance patches that improve its behavior from the divergent clone the hypothetical engineer created.
LLMs output react because react dominates their training data. You have to reject their plan and force them to choose your preferred architecture when they attempt to generate what you ask, but in a different way.
We can have better tooling for sharing apps than the web. First, it needs to be built. This takes effort, iteration, and time.
Second, it needs to be marketed and gain adoption. At one time, Netflix and the <blink> tag it implented dominated the web. Now it is a historical footnote.Massive migrations and adoptions happen.
Build the world you want to work in. And use the tools you think make you more productive. Measure those against new tools that come along, and adopt the ones that are better. That's all you can do.