Software Engineering Is Dead. Long Live Product Engineering

Software Engineering is not dead, damn it. Anything research and original is not dead. Are these things fantastic at connecting concepts, producing digital work products, and putting out derivate hypotheses? Sure. But proper original thought is still firmly in the realm of humans. Though speed of propagation of new ideas probably did change massively.

The line between Computer Science and Software Engineering is pretty blurry and the term "Software Engineering" has been controversial from the get go. Bolting together established enterprise patterns may be dead but creative problem solving with automation and the science of computers surely is not.

It’s not dead, but it’s starting to feel nostalgic. There will always be a few (of us) who still read the code, like the old school “operator” from The Matrix. And maybe the software geeks that still read code will be better for it, but that will eventually be just at the boundaries.

I think for most things by far, product is king. If you're in a deep tech space, then no. Perhaps if you're making rockets, or compilers. But most other things become "make some engineering judgements about what we need to build, based on what the business side is asking for".

Now, it was already like that to an extent. To be crude, the job was "decide what needs to be made, and make it". People would be on various splits between the two, typically on the "make it" end if they were junior.

Initially, I thought about what effect this would have on hiring. Perhaps we would get a lot more people who were not so great at implementation, since we are hiring for judgement?

Let's consider Berkson's paradox.

Let's say that there's a correlation between implementation skill and engineering judgement, eg r=0.5, a rugby-ball shaped cloud that points up and to the right. The basis for this being that you maybe learn good judgement from implementing things, something like that.

Before AI, selection on the sum scores (judgement + implementation) would give you a set of engineers whose correlation was deflated from the underlying correlation. It could even go negative (in fact it would if the underlying correlation was zero, but we assumed it was positive). This is the original Berkson's paradox, selection makes the correlation coefficient lower. There would be less of a connection between someone's (who had been hired) judgement and implementation skill than 0.5, it would tend downwards, possibly landing at zero.

Now, let us imagine that AI means the implementation side becomes less important. We will veer towards hiring people based on engineering judgement, rather than coding skill. Suppose we add a parameter on (judgement + lambda * implementation).

What actually happens is, if lambda tends towards zero, we are selecting on a line that is only judgement. But that means actually, the deflating effect of the selection is lowered, and we get a correlation between our variables that is closer to the underlying set (it won't necessarily be all the way back, since we restrict the range).

So then actually, somehow, people we hire who are good at judgement will also be good at implementation. Somewhat unexpected.

I largely buy this thesis.

I think the space where juniors can thrive in this New Normal is to fill in the gaps in written form. Seniors don't know what we know. I'm constantly finding implicit knowledge I assume a junior possesses that they in fact, do not have an inkling that it even exists. Lo and behold, the loops and harnesses don't, either. That's why they come to me asking the questions they ask in the first place.

Holding that brief junior perspective, filling in those innumerable gaps while learning everything else being thrown at them via the agents, is critical at this juncture for the agents to be able to bootstrap themselves from ever-broader, ever-vaguer initial instructions. When those potholes are mostly filled, I can think of a lot of other stuff for juniors to work upon, but I am sorely missing this piece with agents at this stage of their evolution, and I definitionally am unable to fill it myself. It is the source of most of my rework of their code output.

Software development evolves. Every decade. Or faster. Always has, and always will.

There's still plenty of innovation in the software space. Look at Roc-lang, for example. Fairly ideal as a human-agent collaborative language.