Thoughts on the software industry (2022)
linus.coffee
linus.coffee
Learning for the sake of learning, being able to be picky about what to learn, feeling like a god by creating your own worlds, trial and error… these are all traits of programming. On the other hand, software engineering is about sprints, useless deadlines, using the wrong tool for the job, writing code to increase shareholders value, high-performing-team obsession, useless C-level execs, broken tech interviews…
Two different worlds.
Rather, this is about capitalism. You want more freedom in going about your business. You cannot have that in a profit-oriented company. (Rare exceptions apply.) We're so focused on delivering, what we deliver becomes less of a concern, even though it should be the number 1 concern. The rest follows.
Well, its about what you measure.
A solution oriented programmer can have the best business goals in mind by not half assing a hotfix but proposing a full new featue with large budget demands.
Some managers respect and care about good software but may need to compromise on budget cuts from higher up.
I have seen it so many times.
That is not at all what "Software Engineering" is. It is Management's view and practice of what it should be based on the fad of the month/year, simplistic efficiency/productivity models, cluelessness about human psychology and its specific nuances when it comes to knowledge/programming work, erratic/idiotic business goals, bad human resources management and in general not understanding that while one can look to established Engineering disciplines for inspiration one cannot apply those processes/techniques blindly to Software Development since it is so very different.
The classic works of Gerald Weinberg, David Parnas, Barry Boehm etc. gives one the proper viewpoint to have towards "Software Engineering".
His research was a product of his times when programming/coding was considered a clerical job with the real brain work being done by System Analysts and System Designers who produced System Requirements Specification Documents and System Design Specification Documents. He was the first to show that software development costs would outstrip hardware development costs and the importance of its Risk Management through a Spiral Model. He brought "big picture" engineering process lens to the management of software development which became "Software Engineering" today.
All of these ideas are still valid though they would need to be modified for current times given the decades of experience we have had since then. Software development at the Individual level involves creativity/art/craft/individual flair and companies must acknowledge and provide means for expression of these psychological needs. OTOH, Software development at the Company level is all about Risk Management in the service of its Business Goals and hence needs a reproducible process and associated methodologies. We need to find a balance between the two.
But I think pretty much any business has risk management processes that involve domain knowledge, and a much larger and more distinctive part of any kind of engineering involves non-business considerations and tradeoffs that AFAIK Boehm didn’t really concern himself with. For example, a civil engineer designing a bridge is thinking about wind shear, traffic volume, vehicle weight allowances, and so on, just as a software engineer is thinking about latency, throughput, error propagation, and so on. To me, these sort of non-business-logic but very much domain-specific considerations are what comprise the core of software engineering (as compared to programming, which I would classify as just getting the business logic correct without any meta-considerations).
This comment is poorly edited, but essentially Boehm’s processes to me are better described as “software business management” than “software engineering”.
One important point though; IMO Risk Management in the Software Industry is qualitatively different than that from other Industries. This is because of the greater play and importance of Human Factors involved and their cumulative non-linear effects. In very few other industries (eg. Finance) can you have a factor of 10x/100x productivity between the average and the best since it does not follow the normal distribution. If i have a Jeff Dean/John Carmack programming or a Steve Jobs salesmanship in my company i can break every engineering rule i want and still come out on top.
Nassim Taleb's ideas on Mediocristan/Extremistan, Power Laws/Fat Tails are all relevant here. Here is a great video explaining these concepts (though we have to adapt and apply it to Software Engineering) : Pareto, Power Laws, and Fat Tails - https://www.youtube.com/watch?v=Wcqt49dXtm8
I find such engineering fun and meaningful. As a software engineer, you understand that is not enough simply to write code that works, the right (or wrong) architecture, for example, can have a massive impact on various stuff like performance, reliability, code-quality, etc.
The BS involved in working for the tech industry as part of a team is another thing entirely.
Chemistry is about understanding chemical elements and compounds and the ways they react with each other to form different compounds. This includes the synthesis of new compounds, to understand the mechanism by which they might be created and the properties of the new compound; accordingly, chemical synthesis by a research chemist tends to take place in small amounts.
Chemical engineering is about the production of chemical substances, reliably, at massive scales for industrial purposes. Chemical engineers elicit and apply the manufacturing processes necessary to synthesize these chemicals by the kilolitre or more, which may be totally different from the processes the experimental chemist uses in a lab.
Similarly, programming is the act of writing code for a computer to fulfill some task, or explore some computational idea. Software engineering is the application of a standard, defined process to reliably produce code at massive scales for industrial purposes (enterprise applications, etc.). Architecture, data modeling, and QA are part of it, but the heart of the discipline is finding and applying the means to organize a large team to produce software that is robust, scalable, and maintainable to support an organization over the long term.
People who fancy themselves artisan programmers chafe at the industrialization and standardization of software engineering. But, as Milt Bryce said, "There are very few true artists in computer programming; most are just house painters".
Granted, software engineering is a pretty ill-defined term, but I'm pretty sure most would agree that agile project management is not software engineering.
"Being able to be picky about what to learn" being programming is also a really confusing idea. Programming is writing programs, it's not some learning philosophy, and it's certainly not some liberated opposite to software engineering.
I guess you're confusing having a job with software engineering, and spare time coding with programming.
That's how you learn programming. It's not a bad idea, but at least for me software development is more about long term issues coming up, team communication, features that create short term value but long term problems. How to organize big piles of code.
A lot of abstractions don't make any sense in a 2-3 days project and you are better of hacking away a script than looking into "properly" modelling things.
My impression is always that as a junior you learn how to do stuff. Then you learn to do complicated stuff. And becoming a senior you learn how not to do complicated stuff.
This takes some of the fun out of it as well. Deploying a feature that is simple and that just works without any issues does not create nearly as much excitement as "saving the company" with a big hack and high risk deployments, although it is much better to not have to "save the company" in the first place.
Some people are more risk-affine, and some are more risk-averse.
The fact that such things work rather kindles a desire to do new interesting, experimental stuff in risk-affine people - something that a corporate environment often prohibits, which frustrates risk-affine people.
Key takeaways:
1. Spending a career building computer software around others who build software, influences your worldview.
2. Programming is a skill (like any kind of craftsmanship or performance art); you learn by doing, not just by reading.
You don’t expect to learn how to play the piano by reading a book, but you might expect to pick up a programming language or concept because you read about it. Really, it doesn’t work that way. You have to read it, and then use it and practice it and make a hundred mistakes, and then you’ll gain the skill to use that concept more correctly over time.
I just don't know of anything even remotely similar to this in programming. Learning how to type fast is the closest analogy I can think of, but being good at typing is almost entirely unrelated to being a good programmer. It would be like a pianist not being able to play chords, yet still being considered a (technically) good pianist.
---
If one develops a good technical understanding of their programming language, and reads a lot of code, they could produce some quite good code in that language on their first try. In this case, it would help to have a very solid technical understanding of first principles as well. Asa result, I really think programmers with very strong discrete math skills, for example, can pick up any idea in programming.
although i'd counter with the idea that it isn't code-writing which we practice continually to master the art, but world modelling: we model the world within the constraints of code in our mind, to become experienced enough to do so reliably, then usefully within different contexts
code is the instrument, not the music
What I find interesting is how much of this knowledge we take for granted once we have internalised it. My wife came into my study the other day and asked what I was doing so I started to explain it. After she ran away I counted up the number of concepts I needed to know to understand the twenty five lines of code I was looking at. There were nearly forty in that short piece of code.
Software is an amazing passion, but a terrible job.
Imagine you love music, play the piano for hours every day, then once you grow up someone pays you a huge salary just to play. You're in heaven.
Now imagine someone who loves classical music, but can't play. You see the fortunes being made by piano players, so you decide to learn. Which takes years of doing tedious stuff like learning scales. Then you get a job playing hip-hop (a music style you abhore) for a band you don't like... Then you discover the business crap of "music business".
Programming is the same. If you love it, it's fantastic. But to those in it for the salary it's a mind-numbing hunt for tiny bugs, days spent searching for a missing comma, wading through the same old crud, doing the same task over and over, locked to a desk, dealing with sub-par teammates, inflexible management, projects that get canceled, crunch time, stupid users (and I can go on.)
The passionate thrive because the joy dwarfs the pain. (Also, because they tend to be -very- good so can largely ignore the bullshit. They're paid well doing something they'd do for free.
As a -job- though, every part of it is objectively terrible. From the interview to the firing. (Go on, write a job description of what you do all day. Now consider that description in light of someone who doesn't actually -like- programming...it's terrible right?)
Naturally there's good money in it. If it's not your passion, then by all means, do it for the money. Find your joy elsewhere, that's OK, but do find it. Because without that joy elsewhere, what good is all the money in the world?
The happiest psychologist I know is many, many times more content than the happiest programmer I know at a corporate job. The most passionate software engineers I know might be able to claim similar joy, but they have overwhelmingly quit their corporate gigs and fly solo.
Yes, I'd agree with this. You make more money in a corporate gig, but if you're doing it for passion, not money, then the money doesn't overcome the corporate shackles.
- I'm happy, but not very
- I'm financially successful, but not very
- I like what I do, but too much
- I'm comfortable, but not too comfortable.
It's like my entire personality, likes, and dislikes have been smeared into a 2-dimensional caricature and propped up by a couple supports, for everyone to see and admire. This sort of existence is safe, inoffensive, and unremarkable.
Love your blog btw, it gives me confidence to be more like myself in my own life.
I read about some research recently where the researcher asked people how much money they earned and how much they would need to feel financially secure. No matter how much they earned they all felt they needed about 50% more.
It seems we are programmed to feel mildly dissatisfied no matter what our circumstances. I guess that is what drives us on.
Practically every person I ever asked told me more or less this:
"Yeah sure, who would not want 100K a month? But I am not willing to forfeit my personal and family life for it and that kind of money always comes with a catch. Nah man. I'd be happy with 20K a month but it ain't ever happening while there's always the next a-hole CEO who wants a bigger yacht than his bros in the golf club."
So yeah, I heard your take many times and I have not seen it out there. Not once.
Or perhaps you just had something you wanted to get off your chest.
Not sure what your response contributes to any discussion though. You did not try to defend or enrich your position.
The hedonistic treadmill does not exist.
It's a feel-good construct, I found.
No, it was an actual paper (unfortunately I can't find it now).
>The hedonistic treadmill does not exist.
I never referred to that in my original answer, although it is related it seems straw man like in this context.
I have done the solo thing and have come to realize I much prefer coding for others. There’s just too much annoying bullshit in running a business and I don’t need to deal with almost any of that as an employee despite getting basically all the same freedom, impact, and influence as a high level engineer.
There’s even a point where things switch from your boss telling you what to do to your boss asking you what to do.
If you're happy with not having a bigger slice of the pie, it's easier to just focus on SWE as an employee and let someone else figure out all of the other stuff unless you really crave the company building aspect.
Not to mention a smaller slice of a way bigger pie can still be a lot bigger in absolute terms. With much lower risk.
How? Actionable step by step, please. Measurable, with before-and-after descriptions.
Step 2. If you are still relying on steps, you did not actually complete Step 1.
If so, quite humorous.
The managers who hired them were not complete idiots, mind you. They knew what they were getting but they are playing their own games: they need the bigger head-count, they want to hire their cousin to oversee these sub-par programmers, they want to be seen as professional when they inevitably have to ask for external consulting when their underlings crap the bed on a regular basis, etc.
But it's mostly power play, I found, almost on a sexual kink level. A lot of people in power out there will hire you if you are meek and can't stand up for yourself. Really weird.
Everywhere they go, only one thing ever happens: an assembly line. They will work tirelessly to make each and every business and department that.
My view is that they want to play and not work to get things done.
I also see it as we are engineers first and foremost and we use a software language as a tool to solve a problem.
I’m also a musician who had the chance to make a living in audio production and nothing is worse than working on other peoples tedious projects, so decided to keep the art I love for myself and have a much better paid career in tech.
I guess it’s the opposite of what you’re saying.
Sounds like you need to find another job!
Nope, not at all. It's people who got sick of being sheep-herded into assembly-line processes. That you don't want to be an assembly-line does not automatically put you at the other extreme (namely the divas who want to only work on pet problems and get paid a lot).
I've thought to myself many times that I still can't believe someone is paying me to code, whether that's an investor or an employer. I happily do it for free even when unemployed, or practically for free for years on end when working on early stage startups as a co-founder. I've often wondered what this must feel like for people who don't actually like any of it, when for many of us it feels like play.
Meanwhile many people happily make a living programming in a mostly stress-free, creative environment where every day brings new challenges and the codebase is well ordered and easy to navigate.
Also, tracking down bugs is fun! I can see how it would not be fun in a terrible codebase and have experienced that side of things, but nobody is forcing you to work at a particular company.
It really isn't all that simple. I felt the same way when I was younger - "just switch jobs yo" - but as I grow up I realize that there are increasingly many external life factors forcing people into an unhealthy work environments which can be outside their control.
I work in a shitty legacy PHP codebase, but I do it for the great money and can accept the pain willingly. It is much easier to tolerate things when you know there is an exit nearby, but I imagine people can get much more jaded when their visa depends on the shitty employer, debts loom over their head, medical costs are piling up, kids require feeding every month etc.
Life is too short to stay in bad situations for a long time. Of course it’s sometimes hard to move/switch job but it’s never impossible and why spend most of your waking life on something you hate with unremitting passion like the OP?
Life is also too short to get evicted of "your" house (that the bank actually owns) when you miss 3 mortgage payments because you left because "life is too short to stay in bad situations".
Wait, it turned out I am not forgiven and just taken advantage of.
So yeah, I am trying. Not easy by any means. But lesson learned, though way too late in my life and career. I'll definitely be doing that going forward.
You hiring?
Because what your parent commenter describes are at least 90% of all programming jobs. Everywhere. And I am being generous by not saying 98% because it's likely closer to that number.
"Many people" could be true only in the absolute numerical sense of the world. Say, 500K out of the dozens of millions.
If you're good enough to have more control over the terms of your indenture then it's a pretty solid career. The complaints you level are the same for any job where you're not trusted with much autonomy.
Now consider it from a perspective of someone who hates the job.
They hate the job, right?
This works for any job.
Very well said! And it is all the Software Industry's fault.
When i look back at the noob me decades ago, i see a young'un full of energy, enthusiasm, curiosity and a hunger to learn anything and everything, knowledge for knowledge's sake, to prove oneself to the world and in general a would-be World Conqueror.
So what happened in the subsequent decades that lead to the current cynical fuddy-duddy? The battle in the trenches of "Software Industry" happened. The battlefield was stacked against him in every way possible, viz; He was told Knowledge acquisition without a Business Goal/Money was worthless, People clearly unqualified in the domain were appointed as his Managers and gave orders, His curiosity was curtailed, He was subject to inane and brain-dead management processes, His self-worth was actively diminished by denying his uniqueness and being explicitly treated as a mere inconsequential cog-in-the-wheel etc. etc.
So the grizzled veteran retreated from the battlefield, worked on getting back that "original beginner mindset", rekindle the passion, and vowing to never again allow himself to be put in such a situation even though he might be poorer for it.
Conversely, reality slowly changes to be easier to model in software. When Computer says no, a user adapts.
Gotta be careful with that attitude even though it represents some insight.
Let's take names. I have diacritics in mine and an appalling number of computer systems can't handle them properly. Should I drop them completely and loose a part of my language in the process? Or deal with more □? firstName + lastName is another, where our idea of naming is forces on to others with a different system.
Dates. The US sets weeks from Sun-Sat and much of US software defaults to it, even when it should know that in my locale Mon-Sun is the default. In some cases I can't even change it. And don't get me started on MM-DD-YYYY.
Processes. Talking to a human is easy. There is leeway. A computer says no. Now I have to conform.
More broadly, we humans do fantastically well with ambiguity. Modelling ambiguity is tricky, after all, computers are extremely fast morons. So that ambiguity gets lost.
It's also one of the least meritocratic industries in existence because most big companies have monopolies and can afford to be extremely inefficient when it comes to software development. The bar is mediocre. You just need to rehearse your leetcode to pass the interview; beyond that, you just need to be very good in a tiny area. You will be so highly specialized in the minutiae of your role that basically you will be useless as a software developer. You better hope there exists an equivalent role in a different large company if you want to have any leverage at all in the market. Startups or any business which relies on efficiency and actual skills will not have any use for you...
Still, the biggest problem is just how easy it is for incompetent people to bullshit their way out of anything and the metrics that are used to measure performance are so objectively horrible that it would be better if no metrics existed.
I've been in this industry for over 15 years. Not sheltered by big tech. Careful what company you join because working for the wrong company can be literal hell on earth.
It is clear to me that this industry is still very new and almost nobody knows what they're doing. Most people who lead this industry are not equipped to do so. They just got lucky and now set the rules which are essentially arbitrary and change all the time because they suck.
It's not like Law or Medicine; these industries have existed in various forms for thousands of years and there is a lot of philosophy behind those. It's relatively easy to figure out who is a good lawyer or a good doctor. With software development, it's extremely difficult for people to do because there are many aspects to consider and almost nobody understands which ones are actually important.
Imagine being a really good surgeon with years of experience and a 100% success rate on your surgeries but the hospital fires you because one of your colleagues was jealous and filed a fraudulent complaint claiming that you don't hold your scalpel in the approved way. Without metrics to accurately measure individual performance, everything becomes political hearsay. It becomes all about stupid immaterial things and all the important stuff is ignored.
I hope this saves some people from suffering. I would not recommend this industry if you enjoy coding. There is a non-trivial chance that you will be a life of pain. Especially if you are skilled.
I have met some top engineers who were so bright, passionate and were pioneers in their area and ended up miserable and brought to the edge of insanity. If you don't have deep social connections to this industry or intelligence agencies or other entities that can give you an insider's edge, stay away.
There are many stories about software devs who ended up retiring from Microsoft or other to be goose farmers or similar. These people are not eccentric; they are rational... And these are the lucky ones.
You don't want to know what happens to those who got trapped in this industry, got their passion for coding completely beaten out of them and can't find any off-ramp.
I want to know. What happens to them?
That this comes with an ugly bouquet full of mental and physical health problems is another topic entirely, and that can fill 10 tomes of books though.
I guess it's not the 80s anymore. Those were the good times.
Are you sure that isn't just imagining green grass on the other side of the river? Politics comes with the humans, not the profession. Occasionally in hospitals it turns out that you are working with literal serial killers too. I doubt those sociopaths would have any particular compunction about murdering a colleague, let alone filing fake complaints.
In the past I worked at a factory for a short amount of time, and did not enjoy it. I also worked for a short time in academia (basically an internship), and while better than the factory, I didn't like it as much as my current job.
(Or is this about monopolies using underhanded tactics to try to shut down high profile projects of their smaller competitors ?)