Current Software Engineers Have No Deep Knowledge (Jonathan Blow) [video]

Such a long-winded way of saying, "Get off my lawn! …kids these days."

Having been in the industry for 30 years and alive long enough to run programs for 45, I can point to innumerable examples of badly written shareware games. Hell, I can give examples of then-AAA games where you learned to avoid certain patterns because they would either crash or grind to a crawl for extended periods of time.

Then there are the business apps that used janky proprietary storage formats and silently lost data, sometimes piecemeal and sometimes in whole. Crashes for a time in many apps was normal. Advice was just to remember to save often so you didn't lose as much when it did happen.

How long did we use RCS and CVS and Subversion before modern alternatives like Bitkeeper, Mercurial, Git, et al. entered the scene? You're telling me Visual SourceSafe was deeper knowledge than Fossil?

We have some of the best programmers alive today. And the worst. That's what happens when the talent pool keeps getting larger. It's not a trend of quality; it's just a furthering of the mainstream.

I get tired of listening to this kind of thing pretty quickly. There's very little here that's falsifiable or of substance, very little to actually confront in an intellectually rigorous way.

But I'll add this, it's especially uninteresting to hear comparisons between gamedev and webdev. The goals and constraints are totally different.

Of course you're going to see a thousand very similar frameworks and a million similar npm packages, and of course none of them are going to be incredibly optimized or performant in the way a video game would be. That's not the goal. The intent is to ship things quickly and iterate on what you've shipped as necessary, so that you can test the value prop and your assumptions in the real world quickly.

It's completely opposed to how video games are engineered, which are delivered in (if not a shrinkwrapped box) individual major releases that might come once a quarter or even year. Video games have to meet strict 60fps performance targets, they often push the boundaries of the platforms they're built on.

So what's with all the fake shock? Why do we pretend to be surprised that the output and the process is different when the goals are different?

Get real. None of this performative whining from up on some pedestal helps anyone, except those who are so insecure about their engineering ability that they need validation from some imagined authority.

I am often surprised by how much foundational knowledge is unknown, even when extensively used at work. I can't really speak for game programming, though I imagine the prevalence of game engines has a lot to do with that. In regular SaaS back end work, knowing how your SQL database works is pretty key. Ruby/Rails devs are now familiar with N+1 queries and how to fix them. It's common for the same devs to not know how indexes work and why adding/removing a WHERE condition or changing an ORDER BY can have such drastic effects.

Since the video didn't get into details of examples of what constitutes 'deep knowledge' (or I missed it), I'll describe it as lacking an accurate mental model. That's what I struggled with first learning git, once I had a mental model that was close to how it actually worked, every new command I learned fit into that seamlessly. The part the video does talk about is cargo-cult behavior, which is very much how the building and adjusting of a mental model is avoided--it's just building a memory bank of problem X, incantation Y. These folk will be the first to be replaced by LLMs. Build your mental models and adjust them!

In absolute terms the number of people with deep knowledge here is probably higher than it’s ever been. The ratio has changed, which is what I think he’s actually lamenting here. He mentions civil engineering; one thing I find myself often bringing up is that software engineering is a brand new discipline. The operating domain of softare is immense, maybe infinite. So I’m less curmudgeonly than Jon is here, we’ve got a barely century or so of software experience as a species, there’s gonna be slop.

I'm not sure that there was an inrush of programmers because "it could make a lot of money". I can't see, "Chad" (yes, I am being kind of offensive — but to make a point) having no knowledge or interest in math, programming — pursuing software development just to make a lot of money.

I don't doubt that a number of students who could have chosen among a number of STEM fields did in fact go the software route because it was lucrative. But these are going to be smart people - not Javascript-only code monkeys that the speaker is implying.

There's a spectrum of engineers that I have worked with.

Some are very, very technical, more like the civil engineers the speaker talks about. They care very deeply about optimizing for performance, care about clean API. And, sure, there are engineers on the other end of the spectrum that are more focused on completing a task with perhaps a more pragmatic (?) focus.

Perhaps it's a good thing the industry has both kinds.

(Myself, I was closer to the latter example above — while I could intuit the efficiency of an algorithm, I never did take the college course that covered n^2, etc.)

I think he's saying that we're losing knowledge of fundamentals which is true. It's because corporations are commoditizing and devaluing applicants with university educations to save a buck. The university educations are where people gain knowledge of the deeper fundamentals of Computer Science.

This resonates, but I think it's overly pessimistic about the knowledge that software engineers collectively have. Actually there's a lot of deep knowledge and experience, just not enough to counterbalance the massive volume of people and ideas trying to get things done in the space.

To use his example of civil engineering, the reason that's a more solid technical discipline is because the difficulties and consequences are more visible and comprehensible to laymen. People understand it's hard to build the Hoover Dam and the consequences if you fuck up. It's obviously a huge responsibility that is taken seriously, and does not attract dilettantes just looking to get rich. Because projects are expansive, massive and local, there's no scalability principal that favors engineers contribution. The only way to get rich off big construction or civil engineering projects is to run massive companies, because the actual costs are large, and the biggest overall factor in success is good logistics and project management.

Software is just a whole different thing because there is no physical locality or even logical locality. Code is trivial to copy and remix in any form, heterogenous executing machines can interact across the network. Goals are arbitrary and can be understood differently by every participant if they even are understood at all. Yet despite this, software can be deployed and made instantly to billions of people at the push of a button. Zombo.com wasn't lying, you really can do anything you want with software. Impact can range from world changing and unequaled wealth to a life spent toiling in obscurity on imaginary sand castles. It's no wonder software is a mess.

Ultimately engineers exist to make things, usually to make money. Any trend in the knowledge required to do so is just a reflection of what sells. This is reason why some web developers barely know any theory, they don't need it to make a thing that makes money. Same reasoning explains why a Game Developer (if the money-senders demand, customers, publishers, whatever) might need to know these things.

Just saying "kids these days" or chalking it up to education or a generational issue is silly, there's simply more software and faster computers, unless you make comically bad decisions the average paying customer isn't going to complain about a web app's performance. If you want to find someone who has this "Deep knowledge" of how computers work you needn't look further than someone who works with any sort of hardware limitation.

The comments here are a demonstration of his claim about there being a lot of people chattering in a “loud room.”

I see the same problem in software testing and I see the same outlook with the same heuristic for improvement: just do better in an obvious way.

The marketplace of software development and testing ideas is highly inefficient. Good ideas float to the top slowly and are easily buried again by transient and desperate fads.

I agree that the irrationality of the market may outlast any individual’s solvency. Still, anyone who chooses the path of craftsmanship and excellence can set an example that has some chance to make the world better.

The first time I wrote a comment like this in public was probably on Comp.Software-Eng in 1994, unless it was on Compuserve prior to that. The loud room has been around a long time.

I don’t think this is limited to software engineers.

I feel like change is happening more rapidly, and there is more being created than ever before.

That makes it difficult for anyone to go deep on anything. We are spread thin across vast landscape of languages, tools, and platforms - with new ones being added every day.

On a similar note, I sometimes think about what it takes to be a movie or music “buff” these days.

I was born in the 80s. Being a movie or music buff back then meant gaining deep knowledge in a much smaller catalogue of work.

Thanks to advancement in audio production technology and access to affordable software for home studio producers (plus the rise of streaming and sharing platforms), the catalogue of music in the world exploded. The same applies for TV and movies. As a kid, it was possible for me to listen to pretty much the majority of my favorite genre (metal) available to me and be a “buff” with deep knowledge of that genre.

If I was a kid today that would be much harder, I might have to stick to certain sub genres or focus on only the most successful artists.

To come back on topic, I used to have deep knowledge of a certain type of software - Flash actionscript. Who remembers that?

These days I just get by with what I need to get the job done.

This is almost all he ever talks about and it's getting kinda tiring.

Man who works on software requiring specific knowledge and who runs in circles of people doing the same laments that the other 90% of software developers don't know or value his specific knowledge

I may not be a Jon Blow or a John Carmack but I've been in programming long enough to think 'bullshit' the moment I hear anyone talking so generally about a field basically born yesterday, that looks almost completely different today.

I don't disagree with any of this, however the lack of breadth of knowledge is much more remarkable. Aside from security and hacking, I find Software Engineers lack bare metal and low level fundamentals in electronics and IT. Go ask your Software Engineer friends if they know how to change their IP address or if they know how to replace a PCIe card in a PC without using an online tutorial.

We’re getting more specialized as a society, and we have to accept the facts that not everyone knows everything anymore