Software engineers are not (and should not be) technicians
haskellforall.com
haskellforall.com
Fred Brooks, in the Mythical Man Month, identified the value of "toolmakers" to the "surgeons" many years ago. I think that model applies here too.
Seems to me that a better path would have been to ask the developer in question to lead the automation project, and generally push them over time to look for, and implement, improvements across the development chain that increase efficiency. The elite devs get to keep on doing what they do, but now the whole boat is lifted, and the dev gets to put some real wins on their resume.
I've been on teams where management very clearly tries to do a good team/bad team split.
This looks like hiring some space cadets to build the "strategic solution" that will magically fix every problem with the existing system (despite being managed by the same senior management).
The existing team is told to just keep the lights on / the new thing is going to be so much better / just give it another year or five.
There's not a lot of upside to staying around because you either end up fired at the end if the space cadets succeed, or if the space cadets fail you end up holding the bag of rescuing them in X years post strategic platform failure. And sometimes a secret third thing - the company being run this poorly ends up having to do RIFs and you end up fired randomly anyway.
(actual technicians, however, for example in electronics or chemistry labs, are generally not doing routine and predictable work; they spend a lot of their time debugging why things have failed, which is the opposite extreme, and they tend to automate as much of their work as they can. so i'm not convinced the choice of term is correct)
neither toolmakers nor surgeons in brooks's model were doing routine, predictable work. (but his model also hasn't been very successful, so possibly it wasn't a good model)
i like smaug's term 'toil'. so maybe we should say 'software engineers should not toil too much' instead of talking about what they should or shouldn't be
After some investigation we found that one line in an xml inside the course zip file needed changing. So, my manager planned how to do it. We'll divide the 40k courses into batches, different individuals would tackles certain batches and it all would be done in about 4-6 weeks...
Well, probably my ADHD brain or just being lazy but that sounded like an awful lot of repetitive work that I didn't want to do (change file extension to .zip, decompress, go to file X, find line with XYZ and replace for ABC, save, compress, change file extension back to proprietary format), so after talking with my boss and asking permission I got a week to try to automate that.
Back then I knew almost nothing about software development but I was aware of bash and python... so I cobbled together a janky script that would likely make me cry if I saw it now, tested it on 1 course, then on 10, then on 100... and then shared with my boss. It all worked perfectly! We still batches of about 1k courses at a time to confirm that everything was working ok, and in the end the job took about 2 days and 1 person (as opposed to 4-6 weeks and 5 people).
I was very proud of that accomplishment and very likely marked my perception of the value of software as a tool to decrease human effort (nobody got laid off or leaved, is not like we didn't have A TON of work apart from that migration).
Anyways, just wanted to share :-)
Re-encode video files to web-friendly formats and upload them (prior to YouTube).
There was a guy whose full-time job was to sit and do that with a Windows GUI.
First approximation was an AutoIt script. Second approximation was with bash and ffmpeg.
All of which is to say that I formulate a spectrum from technicians through engineers, to scientists. Technicians use technology, engineers devise new technologies using scientific facts about the world, and scientists discover new science. All three parties end up spending a lot of time on repetitive tasks, however.
Perhaps what makes computer science unique here is that it’s a science of automation, and as such it actually offers us the opportunity to reduce repetitive work.
Unlike other countries in the US you can call yourself an engineer as a job description without any restrictions, so far too many people have taken on that job description, even if it does not make any sense.
This is one of my pet peeves... in Engineering school it is beaten into you that what you build has to be correct. Correct as in, did you check the poles and zeros? Is this thing going to oscillate out of control and kill people in any breeze? Is this circuit ever going to go into a region where it will fail, probably killing people, and did you ensure you have circuitry that will prevent that? Is every floor of this building rated for the loads, winds, seismic activity, etc.
I feel like in software, there used to be a lot of focus on correctness of algorithms. In undergrad computer engineering we had to take algorithms and again in grad school. That class seemed like the one to teach you to be correct, but it was just in your algorithm design. So much of software today controls hardware that may end up controlling (insert your deadly item here: life support, airplanes, cars, trains, critical infrastructure, deadly infrastructure, etc) that it makes me wonder just how much holistic testing of the "in which ways can this software fail and kill people" has taken place.
Personally I think it would be a better world if software were held to the same standards as other engineering disciplines, and we didn’t treat it as somehow less important for software to be correct just because it’s easy to fix. Things would move slower, but we wouldn’t be “spinning our wheels” nearly as much by redoing work and reinventing wheels over and over. A world where software can be considered “done” when it works and is free of bugs sounds amazing to me.
I do not think that is true. The one thing software allows is a large degree of modularity. In Electrical or mechanical engineering everything can always influence everything else. In software you can have very strong boundaries.
>so it doesn’t make sense to hold it to the same standard of correctness, and prioritize speed of deployment more.
Why? I don't see that conclusion at all.
>>so it doesn’t make sense to hold it to the same standard of correctness, and prioritize speed of deployment more.
>Why? I don't see that conclusion at all.
Because, except in like safety critical applications, it’s ok to get something that works most of the time out the door and fix minor bugs later.
Similarly if the software causes a worldwide outage of critical infrastructure.
In many cases it is imperative that the software be correct from day 0. Just like a bridge.
For aerospace software specifically, a lot. Embedded Software Developers also are often people who also are Electrical engineers and understand how to do proper engineering of a system. In aerospace you have very specific guidelines on how software is developed and tested and which tells you what you need to consider during/before/after the development process. As is common in engineering the "doing" part (e.g. writing the software) is the least relevant part.
I'd wager 60% of software engineers work on teams where they have an existing product, add a feature, and then when done go add a new feature. A sizable % of this folks so have other teams working on other parts of the same codebase. Development is incremental, with no long-term vision, and certainly none of the hard metrics for success.
Speaking to the parent, we had a couple weeks of engineering ethics in our computer engineering undergrad. The dude railed on software for having defects, for being at a crisis point, and I threw back super hard & continue to think that most of the field has to face down enormous combinatorial complexity of inputs. And we have to work against moving target codebases where our added work is typically dwarfed by existing complexity. And unlike a bridge or a floor, there is very little opportunity to overbuild against specifications. We can't shoot for 130% of expected load (in most circumstances), it works until it doesn't.
Thankfully I feel like most mission-critical devices tend to have more limited missions, aren't so open ended, but most software development feels more like an ongoing effort to keep iterating and adding than an cross-the-finish-line effort. Every sprint should be at least one release right? How do we get such high assurances in such regular repeated development cycles?
For me, I sit next to EEs and MechEs all day working on safety critical systems. There are some differences in our jobs, but frankly I don't see the substantive differences that would make one or two of those non-engineering compared to the others.
[0] https://www.hillelwayne.com/post/are-we-really-engineers/
(Obviously the blame for this issue isn't solely on the software side)
Except we allow people to call themselves engineers without having professional accreditation.
Naturally. We allow people to practice engineering without professional accreditation. It would be completely nonsensical to prevent someone from being able to state literally what they do. Some specific engineering areas (those most likely to cause human harm) may be more discriminating with respect to who is allowed to do the work, but with respect to engineering in general it is open season. Anyone with the will is free to do it.
We don't (at least with some assumptions about jurisdiction) allow people to practice law or hair styling without professional accreditation. Anyone claiming to be those things without the professional accreditation is lying, so there is at least some logic in trying to stop people from lying. But not so is the case for engineer. Not having professional accreditation does not imply the same.
Except in my country we do prevent engineers from calling themselves engineers unless they have professional accreditation, even though we quite happily allow engineers to practice engineering without professional accreditation. It's the stupidest thing.
You may have a point that the prevailing idea of what engineering is does not recognize software as an engineering discipline as it does not fit into any of engines, machines, or public works. But if we were to include software, surely all software practitioners are people who design, build, or maintain software? Even the web API guy from the story is an engineer on that end.
If you look at what engineers are doing in other disciplines you will find that in software there are some people doing the same thing. They are doing things like drafting requirements, designing, defining, simulating, overseeing, but don't spend much, if any, time actually building things.
Which is design; captured in the definition.
You're welcome to invent whatever meaning for engineering you want (although you kind of need to define it, in that case – we can't read your mind), but going by what most people consider engineering to be, either all software developers are engineers or none of them are.
Why? I have worked jobs where I definitely didn't do any real design and was just there to implement certain things. I absolutely wouldn't call what I did engineering.
What is the significance of you not wanting to call it engineering?
It conflates two separate things with not a great amount of overlap. It also describes two different paths, an academic path and a tradesman path. The distinction is obviously useful in describing people/roles/activities.
You could also ask why we are conflating machinists with engineers, clearly machinists are building things, definitely more so than engineers.
You could, but it would be rather silly as machinist is clearly a subset of engineer (within what most people deem engineer to mean). It is not a conflation, it is a more precise term. Like using "surgeon" over "physician". It is not like a surgeon has anything to do with your family doctor evaluating your common cold symptoms. Those are entirely different jobs too, yet absolutely get grouped together.
The problem here, it seems, is that we've never come up with generally accepted terms to differentiate the different roles under the software engineering umbrella. I expect that is because the differentiation doesn't matter beyond trying to appeal to some pointless emotions.
That is ridiculous. Engineering is distinguished by being an academic career.
Obviously there is value in distinguishing academics and tradesmen.
I expect you mean that Professional Engineer (PE) is distinguished as being a member of a certain professional organization (or groups of organizations). Indeed, that is true. Has little to do with the topic at hand, though.
I'm not familiar with any differentiation by academic career, unless you are thinking of "Professor of Engineering", or something along those lines? But "professor" seems to be the operative word there.
What about the definition of engineer even suggests academics? It is, in my mind, decidedly pointed to practitioners. It literally states "design, build, or maintain". Those are decidedly not academic pursuits.
It is pointless to discuss here, if you do not see the value in distinguishing the career of learning a trade and a getting a degree, I won't convince you otherwise. Obviously the rest of the world doesn't consider a bricklayer an engineer, just because he is building something and wouldn't want to conflate the civil engineer responsible for that building with the profession of brick laying.
Same here, but we're still going to call other people engineers because usurping a term already found in the common lexicon and trying to hold it legally hostage is the dumbest thing I've ever seen. The existence of a law does not imply sound reasoning.
> if you do not see the value in distinguishing the career of learning a trade and a getting a degree, I won't convince you otherwise.
If there was such benefit, professional engineering organizations would serve no purpose. In reality, it's the professional organization that brings benefit. After all, you can't take someone's degree away. But you can remove them from being a professional member when they don't abide by the "engineering code", which is where the actual benefit lies.
But if you want to call attention to the degree you hold for whatever arbitrary reason, why not simply say "I have a degree"? Why would "engineer" need to say the same thing? In reality nobody is going to care anyway (they might care if you are a member of a PE organization, though), but if they did for some bizarre reason, they're going to want to see proof, so your word doesn't matter anyway. There is good reason the prevailing definition of engineer does not imply anything of the sort.
But, as least as far as I know (and nobody seems to have anything to suggest otherwise), we've never done the same within different software engineering specialities. The only thing that I've ever seen that might come close is "software architect", but despite working in the industry I honestly have no idea what that actually means. The people who claim that title don't seem to do anything different than anyone else. Without a "software brick layer", what could it even begin to mean? Whatever it means, I'm certain the average person on the street will have no clue as to what a "software architect" is and how it differs from anyone else working on software.
As such, what I suggested is that within the software discipline we've never reached generally accepted terms to spell out that differentiation because it doesn't matter in software. We get a few people here and there with bizarre emotional attachments to different jobs that wish there was differentiation (what I suspect is the source of "software architect"), but that does not make for a practical reason to actually put in the effort on a population scale.
Sure, perhaps it will start to matter as the field evolves. Software is still quite young in the grand scheme of things. The differentiation between brick layer and architect in the early days of building construction was no doubt equally useless; only becoming useful as the profession grew up. And, indeed, when that time comes we'll have little choice but to start to settle on words to describe different software development jobs, but until then...
If you are a professional, the norms of your profession override your loyalty to your employer. Your boss cannot override your professional judgment, unless they are also a member of the same profession and willing to assume responsibility. And if something goes wrong, it can be the professional who will be held liable rather than the boss.
“Software Engineering is the part of Computer Science that is too hard for the Computer Scientist.”
However, I think if the tech industry adopted this distinction formally, the outcome is that there would be far fewer software engineers and far more software technicians.
Which would probably actually be a good thing overall. Most solutions being built out in the world don't seem to need software engineers imo
Sometimes people too good get bored and need to invent more complicated versions of the actual problem in order to build fun projects.. All good at your own startup or in your free time of course...
Sometimes 10x engineers means 10x more complicated than it needs to be an 10x less intuitive than it should be. :)
These things can run illogically longer than you can tolerate an unpleasant work environment / bad comp.
Like optimization, there is a sweet spot.
I think it’s a good idea to have a mix of both. There is a certain amount of toil involved in large systems and automating every single task is not always worth the effort. It’s nice to have technicians around to help with that toil.
There are still a lot of folks here who will call themselves, "software engineer," or have it in their job title. However the Professional Engineers of Canada has successfully sued people in recent years. Some cases were defended successfully but various engineering boards are appealing them:
https://globalnews.ca/news/10163565/alberta-engineers-appeal...
https://engineerscanada.ca/become-an-engineer/use-of-profess...
Myself, and like minded people, use other terms like "Software Developer," instead. Personally I would like the chance to be licensed but I lack the education requirement and it seems like, in my province, they removed the equivalent work experience route.
Also... these boards haven't been targeting companies lately for using the term and practising engineering without a licensed engineer on staff supervising anyone... to my knowledge at least. Software might be a different world if we were beholden to our profession rather than our employers.
As a fellow developer with no formal education I also feel that strictly requiring a CS or similar degree probably isn't the best way - that there should be a way to account for work experience, perhaps some kind of test. Wouldn't mind taking one myself.
The further I am in my career though the less any employer seems to care where and what did I study.
However, balancing this consistency with new adventures is where things get a bit tricky. I kind of like Jeremy Howard's approach to learning here. I don't know if it can be applied to a company or a team scale.
Spend 50% of your time on predictable tasks, those that you have mastery over and can do comfortably. Spend 50% of your time on frontier stuff things that break your comfort zone.
Over time the some of the latter tasks will get into the former category, thereby leading to automation organically.
The ratio of comfort:frontier tasks is personal and let's engineers choose their ratio. Some may want the ratio tilted in the comfort zone (greater predictability ) while some may choose to adventure into the frontier zone (lesser predictability ). An organization should have space for both. Even the same individual can alter these ratios based on their life stages, external circumstances.
Put rather simply, a good software engineer can also choose to be a good technician and vice versa. Why should these roles be mutually exclusive ?
I think it's terrible for the knowledge worker, causing both great emotional turmoil (at the dissonance of expectation versus how they actually perform) and reducing quality of output greatly (by forcing output when more dweel/pondering time is required).
Hammock Driven Development by Rich Hickey was such a an enormous relief of a talk to run into, back in the day, hit upon so many tensions & frustrations I had because business Taylorism expected consistent output, expected me to know the process from a->b and to be able to say at every point I am x% of the way through the journey. The businesses doesn't know what's happening but cares enormously about making sure it's extracting as much consistent "progress" as it can out of me/us.
HDD talked to how analysis & understanding, how development & problem solving iteration really work. And it works nothing like what a business wants. And what a business wants hurts the quality & doesn't permit ideal outcomes to emerge & surface. https://github.com/matthiasn/talk-transcripts/blob/master/Hi...
And the human factors are that dwell and pondering are essential for creative and knowledge work. Time in the hammock opens possibilities, fuels the muses.
> Spend 50% of your time on frontier stuff things that break your comfort zone.
That would be so excellent. I still get that mandate a lot, but so often I see peers whose whole life is cranking out pre-digestrd ready-to-go tickets. I want to see a world where we raise up engineers & engineering, not just ticket technicians.
Of course as time goes on we get better automation tools. what used to not be worth automating often is today.
Don't forget that humans tend to be good at noticing "thats funny" situations while automation will pass many of those situations because you didn't think about them and thus didn't handle that error. This is why despite being a big deliver in automated tests I still demand a large amount of time spent in manual testing everything.
That is a pure and virtuous stance.
I typically refrain from giving management ideas of this sort. It's not a given that the pros outweigh the cons - especially in large organisations.
In any case I'm currently in a project which was supposed to be for Technicians, but I need to do a lot of Engineering due to, well, bugs and tech debt.
My take is that any sufficiently long running project, even if originally simple, creeps beyond the skillset of Technicians.
The technician has to know the system well enough to respond correctly (or even proactively with preventive maintenance, etc.) to unpredictable problems that are "in the book".
The programmer by definition solves novel problems.
- - - -
edit to add: Beware folks who look down on technicians. It's a kind of "code smell" of the mind.
In the original blog post, the developer who was automated away was of that type - adding HTTP API endpoints from protobuf definitions if I read correctly.
The point I'm making is that that's not what makes a technician (that's what makes a human robot! Ew!)
> It means that you don't need a technician at all where the problem is completely automatable.
You can try to make your systems so good that they seldom need technicians but you can't fire them all because Murphy's Law rules real world systems.
My thesis is that this is primarily because the normals picked on the nerds up until they realized there was a lot of money to be made if you're the right kind of nerdy (or if you can fake it), around the time of the Dot Com Boom. There's residual resentment there.
Also, people who are otherwise intelligent but unwilling to learn to program computers themselves seem stupid to the dedicated computer nerd, no matter how intelligent they are.
I'm not making excuses, these are just theories about why we nerds can be so ... arrogant?
It's possible to do this without writing code. https://github.com/grpc-ecosystem/grpc-gateway
You can even get it to generate Open AI style documentation.
Being so special you are exempt from having to write documentation, tests, give visibility, etc.
Automation is can be partial, like series of scripts that are ran manually.
I think the article misrepresents what actual technicians do and their value.
Both extremely important to the business but sysadmins are perceived at a lower level?
Intelligent & Lazy: I make them my Commanders because they make the right thing happen, and find the easiest way to accomplish the mission.
Intelligent & Energetic: I make them my General Staff Officers because they make intelligent plans that make the right things happen.
Stupid & Lazy: There are menial tasks that require an officer to perform.. and they follow orders without causing much harm
Stupid & Energetic: These are dangerous and must be eliminated. They cause things to happen, but the wrong things, and so cause trouble."
i find this article to be overly long