6,390 karma · joined February 26, 2011
Book: The Art of Agile Development (now in 2nd edition): https://www.jamesshore.com/s/aoad2
Blog: https://www.jamesshore.com/v2/new
Other stuff: https://www.jamesshore.com/v2/best
Contact: jshore at jamesshore.
Mastodon: @jamesshore@mastodon.online
I’m sorry all the airchair geniuses in this thread feel compelled to express how they’re so much smarter than you and would never fail… or at least, never admit it.
(That and the normal herd of grifters who pile on to every fad.)
It’s been true for a long time. One of the hardest things as a senior leader in software is dealing with people demanding “accountability” (by which they mean making long-term plans with impossibly precise forecasts) and focusing on costs, all while ignoring value and refusing to engage in prioritization. (I swear, if I hear “it’s all important” again…)
People are just… shallow. They operate on feelings and vibes. They follow the herd without thinking critically. Then they get angry when their dreams clash with reality, and they blame the messenger when those dreams turn out to be fantasies.
But you’re right. AI is bringing out the worst in these tendencies. I think it’s because it’s so convincing when you don’t dig deeply, or aren’t an expert in the subject being discussed. On the plus side, it raises the floor, but I think we’re in for some difficult times before the lessons are learned. I don’t think it will take long, though: I suspect that naive use of AI is going to massively speed up the technical debt curve, and where it used to take 5-9 years to destroy a codebase, it will now take closer to 1-2.
It starts off great, with a compelling story about responding to a roadside emergency, but then it veers off the interstate and meanders through the countryside. The paragraphs — they compound. The em-dashes — they multiply. Words upon words and I still don’t know what the author is trying to say. Something about a system for responding to corporate pseudo-emergencies. I guess.
I’m a writer. I get what a preface is for. And I get why an executive coach writes a book (and it ain’t the royalties). But please, please: hire a development editor. Preferably one not named Claude.
(PS: emdashes are supposed to be used—as a copyeditor once told me—without spaces on either side. It’s one of the ways you can tell the true lover of the emdash from an LLM.)
Instead, I had a career ladder with a detailed rubric describing the skills an engineer at each level was expected to have. (Including communication and peer-leadership skills.)
Managers performed qualitative assessment of employees, using the career ladder as a guide. They relied on tech leads and Staff engineers to help them understand people’s skills, and provided 1:1 feedback and coaching.
We did use impact-based metrics to assess the results of important initiatives. We solved the attribution and lagging indicator problems by estimating impact rather than measuring it, and using a series of proxy measurements (activation, usage, retention, etc.) as a feedback mechanism for revising those estimates.
For design, Evans’ Domain-Driven Design does a good job of representing classic object-oriented design, and Fowler’s Patterns of Enterprise Application Architecture is good for expanding past business logic to larger systems.
For coding habits, “The Pragmatic Programmer” is a classic for a reason.
Martin and I have our differences, but this article isn’t really useful. Martin has a brand and a style. A lot of people find it engaging and entertaining. If you don’t, that’s fine, there’s plenty of other ways to learn about the ideas that are more concise.
It’s not human, of course, and I think this problem actually relates to the fact that LLMs don’t have a world model. They don’t study and think through a design in the way that humans do. They don’t form a mental model of how everything fits together and how that design can be tweaked to most elegantly support a change.
I suspect that this is a fundamental limitation of LLMs, and that design will remain a weak point until some sort of bespoke design AI is bolted onto the side. In the meantime, we’ve got a lot of people producing a lot of code very quickly, and I think the debt in that code is going to be a millstone around our necks for a long time to come.
For example, the project we were working on was to add support for reading a session cookie to a codebase that, up until now, had used a different kind of auth. Fairly straightforward, everybody knows what a session cookie is and how it works. In about 10 minutes, we decided on the big picture design elements (how it was going to fit into our existing system, what we needed to add/modify, etc.) and the corresponding tasks.
One of the things we wanted was an “UntrustedCookie” class to represent the cookie. It was meant to follow a pattern we had already established for other user-controlled input. Our HttpServerRequest object was going have a new getCookie() method that returned it.
This would have been about 30 min of work for a pair to implement, including tests. It’s pretty trivial. No further documentation is needed.
Anyway, I’m glad AI is working for you. My experience is that it often fails, and does so in ways that experienced humans don’t.
The statement was that AI is as good as the “best human programmer” and it’s quite obvious that it’s not. It makes inhuman mistakes on a regular basis because it’s not using human thinking. Blaming those mistakes on poor management is just sweeping the problems under the rug.
I don’t know the best way to work with AI, but I do know that we’ll only discover the best way if we’re honest about its capabilities. That includes not pretending it’s as good as the best human programmers.
It’s really not. Opus 4.8 can’t produce good software design and it still makes straightforward implementation mistakes. Two errors it made in one day for me recently: it built the Cookie class I asked for without a name field—cookies have a name and a value—and it neglected to handle a case where a database could have multiple rows with the same id, just returning whatever came back first.
The “best human programmers” absolutely would not have made those mistakes. At worst, they would have asked if I really meant what they thought I meant.
Yeah, citation needed on that one.
I haven’t read the article, so maybe it’s been covered, but here’s a simple way I usually “ask for no.” I send a message on Slack:
“Hey (name), I’m planning to (do a thing) on Wednesday. Let me know if you have concerns.”
It doesn’t come across as usurping authority, or sneaky, or any of the other very italicized things you’re worried about. It comes across as polite and confident.
I’m a VP and this is how I expect my managers to interact with me on major decisions that are within their purview (I don’t need to hear about minor decisions) and it’s also the culture I’m trying to create at the team contributor level as well. Ownership and autonomy, within well-defined guardrails.
See also Turn the Ship Around and its “I intend to” structure.
Personally, I’m not convinced by the premise: I don’t think a wealth tax will significantly impact business formation. The author takes it as a given.
(I’m not a member of the community, so not fully aware of the dynamics.)
What makes Star Citizen unique is that it was crowd-funded—it started taking money before any features were available at all, and released to buyers (“backers”) in an unusually incomplete and buggy state. But all that revenue is turned around and plowed right back into development. (Their finances are publicly available.)
The game remains incomplete and buggy to this day, but with lots of stuff to do and clear forward progress, and, from what I hear, it offers a uniquely immersive experience. People like it enough that revenue keeps increasing, with nearly every year setting new records.
So that’s all this is: a moderately successful company with big ambitions, a lot of bugs, and a passionate fan base. Its annual revenue is nothing special. Don’t fall the sensationalism of the way its modest lifetime revenue numbers are shared.
(Just an interested observer with no horse in this race. I’ve never played the game or spent any money on it.)
One major weakness of this study is that they didn’t fully test frontier models for cost reasons, so the specific performance results should be taken with a grain of salt. But the overall conclusion that models degrade when both behavior and architecture must be correct is interesting, and something to keep an eye on.
Is this just a rhetorical flourish? I’m not up on the details, but it seems like Musk just screwed things up and walked away scot-free. What path do you see for him actually being held accountable for the damage he caused?