The Two Classes of Software Engineer
qvault.io
qvault.io
This is how a lot of school works and I hate it - they teach the abstract, generalized case first and only later do students apply it. They seldom get the experience of working through a practical problem like “my code runs like dogshit when n is a million” and then learning big-oh when it has context.
I need to know why I should care about something to learn it properly - I need concrete motivation. Otherwise.the lesson is just a bunch of facts, as dry as memorizing a phone book.
I generally prefer to tinker first, then learn theory and I’m fairly sure I’m not alone in this.
I started out majoring in Computer Engineering, an embedded systems oriented curriculum. They started us with assembler, then C, and worked our way up. I switched over to Computer Science, which came from the other direction: Java, then C, then assembler.
I learned in myself and saw in my peers that both ways can work great, just for different people.
I struggled to understand how to apply C's pointers and memory management in homework without the context of how that fit into solving a problem in a program, because we hadn't yet written code that really did anything, just one-off tiny functions that demonstrated a mechanic without any purpose. Class tests felt like writing an essay about the applicability of the 4x2 Lego brick without having ever assembled a Lego set. I needed top-down learning to grasp the "why" before I could tackle the "how".
Some peers expressed the same difficulty as me, but others commented that they found their Computer Science courses challenging until they dug deep enough to understand the underpinnings of the computer. They struggled with their Java courses, but got much more confident once they progressed to C and then assembler, because all the unexplained high-level incantations suddenly started making sense.
Over my career, and in my hobbies, which of these approaches I need has come to vary depending on the task. When I'm learning a new way to do an old thing, I jump straight for the bottom-up information. If I'm learning something totally new, I still need to go top-down.
Not in my experience. It's a system of interconnected pieces that often build on each other. With each piece you usually get some justification and relevant problems to solve. That's very far from being just a list of facts to memorize.
I think the educational ideal is something more like: 1. Teach how to do something specific, e.g., "here's how to write a simply dynamic web page in Python". 2. Once they've done that, guide the student into a problem, e.g., "what if the user puts a <script> tag in their username". 3. Solve the specific problem (encoding user input properly). 4. Now, contextualize that problem in the broader space. ("this is a pervasive problem in protocols, let's talk about parsing theory and show the mathematical way in which this is fundamentally ambiguous".) 5. Finally, show another instance of the problem to show that #4 wasn't just an academic thing ("remember the $NEWS_ARTICLE about $BAD_PROBLEM? That was a similar encoding problem only in email instead of HTML").
I think pretty much all those steps are important, all the way out to #5. You can tell #5 is often not done by how many people manage to walk out of a 4 year computer science or software engineering education convinced they've learned nothing of value, because none of the actually-quite-substantial amount of stuff they've learned was contextualized properly.
What schools teach CS101 this way? At mine, the first course was taught with pseudocode so I can't say that was different than your description (but they dropped it after my year and went to Scheme and then Python, not sure what they use now). But the 2nd and 3rd courses had assignments where they threw absurdly large inputs at the program to demonstrate deficiencies in algorithms and data structures at different scales. In the 3rd course, this included not just "theoretical" issues around big-O, but also practical issues with how memory and CPU cache work (access patterns of multi-dimensional arrays in nested loops, in particular), where you were still doing a O(n^2) algorithm but just changing which loop referenced which index led to a noticeable improvement in performance.
One of the neat things about most applied CS is that you can teach it hands on, with minimal equipment. Even more, you can teach it using "toy" or "educational" examples—Minix instead of a "real" OS; algorithms in Lisp or Haskell instead of Java and JS.
I went to one of those universities that taught all the freshmen classes in an obscure language, and that bounced between different languages and platforms throughout the curriculum. While plenty of students complained about not learning immediately marketable skills, I thought this was great--it cut down to size the kids who thought they knew it all because they could get by with common tools; it abstracted the problem space a bit to be more of an exemplar (but, to your point, an applied example); and it stressed the importance of knowing _how_ to think, not just how to use one particular tool.
I look back on my college days, and had maybe 8 classes that were actually useful, and that’s being generous. Yes, I was a STEM major.
I liked the idea of Bootcamps, but the quality is so spotty.
I look back at my instructors in college, and if not for that 10 year, so many were basically useless windbags. Zero practical knowledge, or if they had it kept it secret.
Computers, and a Programming, is still so new that a determined student can possibly do so much more than the guy who’s just taking classes because they feel it’s a safe career.
(That last paragraph only applies to guys under 35 though. I have never seen such ageism in any industry, with maybe the the exception of porn.)
It would be more productive to simply assign attributes to people, and allow all combinations of those attributes.
For example, there are people who have a formal CS education, and those that don't. There are those that have a good grasp of data structures and their applications and those that don't. Those groups have overlap, but not 100%.
If you want to argue that people should have some particular fundamental skill, say that particular skill instead of an abstraction (A CS degree, or 'fundamentals'). We have all met people with CS degrees that couldn't solve a simple CS problem, and we have all met the inverse.
From a practical perspective, you need both the fundamental skills that a task requires as well as the skills to use the tools.
[resolved]
found the submarine
A submarine would be a "news" publication with an article saying e.g., "There's a massive trend [you need to keep up with], to teach CS with approach X. One user of approach X is QVault, which is super successful..."
This isn't doing that. It advocates something, as the source, provides insights, and has a small "ad" at the end that discloses the real source and motivation.
I'll be honest my education doesn't seem to have been very important for the actual work I do. It has been something on my resume that gets me past a basic level of hiring gatekeeping for every job that I've had as a dev, though.
In my experience I've worked with a lot of devs from different backgrounds. Other CompSci majors, Self-taught, Bootcamp grads, and Software Engineers (As in actually graduated from a Software Engineering program).
The approach to software that the Engineers take is equal parts impressive and annoying to me. They have formalized so many things that I know and understand intuitively, but cannot really converse with them because I don't know (and don't really care about) their formalized language.
It has really made me realize that even with my degree, I'm more or less self taught when it comes to writing software because as people rightly point out, CompSci is not about writing code.
This might have been the case decades ago, but it isn't anymore. Every CS program has "applied" courses for students to learn practical job skills, i.e., iOS development, Intro to Cloud Computing, modern web development. These courses are usually pretty good and are sometimes have a corporate sponsor.
Plus, internships are highly encouraged, or sometimes required by programs. So finding a job after (or before) graduation is easy.
> The difference is that people that started on a “computer science” path will have an easier time learning React, Django, Rails, you name it
I always felt the difference is that CS teaches the skills necessary for jobs 5-10 years into the future. For example, it's easier to understand TypeScript when you built a primitive version of a JS trans-piler as part of a course on language theory/design.
When I was in school, nVidia's CUDA had just come out. nVidia sponsored a course to teach CUDA at our university, complete with a lab featuring top-end PCs with the best video cards they offered and a specific book. Practically speaking, most of us who took the course did so to play games on these nice PCs. But CUDA has beccome dominant in the industry, and I suspect that's because they got to teaching it in university early.
Edit: It’s also captured in something the professor said in the MIT intro to electronics course video, while working an example of how a digital logic gate works. Paraphrased: You may be thinking, why do I need to know this? Don’t I just buy these as chips and wire them together? Well, you’re at MIT because you want to be the one designing the chips.
There’s a whole spectrum of positions and titles based at and in between those classifications depending on the organization one would work at but that’s the usual structure.
I _do_ think there are two (or more) classes of developers: there are clearly those who bounce among the FAANG, might become senior staff or engineering leadership, and, yes, tend to be worldly, well-read, often went to top universities.
Conversely, there are "developer" jobs where the interviewer asks if you have ever used $technology_of_the_day, where the opportunity for growth is limited, and "code monkey" is a pretty apt descriptor.
That's, unfortunately, pretty accurate, in my experience.
Is big-O notation important here? In my experience, not really. But _knowing_ _how_ _to_ _think_ is. Unfortunately, big-O interview questions are a not-very-good filter for "can you think", just as top universities, working at some _other- FAANG, etc, are not-very-good filters for that.
It sucks. I don't like that this essay entrenches one of those crude filters. But the reality it describes is probably not too far from the truth.
Anyway here’s a link to the comment: https://news.ycombinator.com/item?id=12079697
In truth, though, I always felt that the large size of "group 1" was an indictment of "group 2".
Put differently, I sometimes joke about big companies like LinkedIn or Facebook: "Why does it take thousands of engineers to run a website?" ;)
Most EEs and CS folks have told me not to study it. They said it’s not necessary. That it takes too much time and money given the fact that I’m already a “software engineer” with a few years of experience.
That said, I could see how it’s useful for filling knowledge gaps & a signaling credential for being a technical leader (ie CTO).
Still, many courses in CS seem irrelevant to what I do. Though again—- it’s cool knowledge to learn. I just don’t deal with networks, typed/compiled languages, assembly, and certain other fields in my work.
If I did study those areas though it would open more doors to me & I’d be able to work on more complex projects (such as web apps involving network work).
These days, now that I understand server admin & programming, I want to learn electrical circuitry, in order to safely control motors, ICs, and machines, via programming. EE degree necessary here? No, look at at Lucky Palmer. However some courses would be useful.
And if I want to be an upper level technical executive I think I’d do well to get a CS or EE degree (depending on my goal of whichever industry)
Just my two cents.
That's still a thing for sure, but the real distinction is between people who can learn anything and people who prefer not to. We've all met people who don't care how their phone works, they just want to know what to press. And a bit more rare is that friend who takes the phone apart, writes apps, and knows about the supply chain.
That last guy is also the guy who bothers to learn the "fundamentals".
So this is the real spectrum, person who checks a few things vs person who has followed all the rabbit holes.
You'll find similar spectra in other walks of life. There's traders who know how options work, and there's guys you can throw a new instrument, and they'll tell you how it works. There's chefs who can make a perfectly decent omelette, and chefs who can make you anything from any cuisine.
I'm would never consider myself a software developer, but I write code all day long and make a very good living doing it. I don't know b-trees. I don't know the algos. I DO know how to think outside the box, to make a creative solution that meets multiple needs, that often exceeds expectations in surprising ways, and blends together the best thinking from several different domains.
I have worked with what I would consider "real" developers, but while I would consider many friends, I don't like how they think, I don't like how they react to new information, I would hate to do their jobs.
So where am I on this spectrum? I don't know. I see absolutely zero need to study the things TFA talks about. I would rather study entirely different domains to bring that information in than learn the minutiae of a linked list I'm never going to implement from scratch anyway.
Just my two cents.
I've been giving what you might call "enrichment" lessons to a friend in lambda school, and half an hour with a makeshift graph in Microsoft Paint ("Quadratic: BAD. Constant and logarithmic: GOOD. Linear: STILL PRETTY GREAT") was all it took. Like, sure, he's not going to be coming up with new compression algorithms any time soon, but...
IMO the stuff that's harder to teach (and to learn!) are things that impact development velocity---how to write maintainable code, how to choose good design (and avoid bad!)---basically, taste.
Software is a team effort, and process scaling is a hard problem. There are software techniques and principles related not just to efficiency, but to protecting the code from the developers in a sense and to be able to scale the engineering team.
There are error reduction strategies that make code more resilient in large code bases and easier to update. These strategies, when used properly, help fight Conway's law and a pure CS background usually doesn't focus too much on them.
Understanding and managing risk, decision making in the absence of perfect information, etc. are skills that engineers are trained for that I've found that pure CS backgrounds struggle with. They eventually learn it, but it takes them a couple of years to grasp that the problem is there in the first place (to be fair, even trained engineers struggle with the finer points).
In any case, whatever your background you still need to be able to handle the trifecta:
- business
- science
- tooling
if you really are to excel at it and regardless of your background to keep perfecting yourself is sound advice.
All those wimpy compiler, algorithm, programming, etc courses got swapped out with electronics courses. I EAT feedback loops like they're spaghetti! My final design project wasn't some weak software project. No, I built the hardware THEN wrote the software for it.
I guess I am in my OWN class.
// This is satire in case anyone needs that put down explicitly.
Articles like this are fun to read sometimes, but don't take them too seriously.
There's a tier that sits in the middle who write "platform" software on top of "systems" but for "applications", but you could argue that those people are much closer to app devs than system software devs so they may as well be lumped in with them.
There is simply less supply of systems jobs and they have a higher talent requirement, so it's pretty rational for people to be content to stay at the application dev layer.
Where is the evidence that systems devs get paid more? My impression is that lots of systems development historically has been unpaid or paid via open source foundations. And the route to getting paid doing that kind of work seems a lot more bumpy and less straight forward, almost like a "starving artist" kind of life story.
Whereas app dev is pretty simple. You go to college, you crank on LeetCode for a couple of weeks and then you get a 6 figure salary working on JIRA tickets. It's a brainless, hard to get wrong life script.
I have a Master's degree in Computer Science but that doesn't necessarily make me better than someone who doesn't and it certainly doesn't mean they are lazy. It's a different path to the same objective: solving problems.
My advice though: stay away from the condescension, I don't think it'll get you a lot of customers.
Most of the wheels have been invented already. Today's high level of developer compensation is a result of business potentials rather than the need for hard science like rocket science.
Once we reach the point of saturation, either due to reduced business demand or increased supply, or both, most of us can't maintain the same level of income. There is a small subset that still can but it will be hyper competitive and can't take in that many people.
Don't let the focus on credentialism make you feel second class. Most of the high honors CS grads I've worked with just prefer work assigned to them in sprints and to crank out achievements endlessly. That's great that it works for them.
What's harder to find is people who pursue an interest in advanced topics that aren't dealt with in CS programs. I can count on one hand the number of engineers I've worked with who didn't need an explanation of CRDTs or CQRS or who've looked into things like TLA+...I'd say most I've worked with are light on the "how X protocol works" side of things as well. Implementation details are our wheelhouse.
"There are 2 types of software engineer: those who understand computer science well enough to do challenging, innovative work, and those who just get by because they’re familiar with a few high level tools."
I get the feeling that is where this vast divide comes from. If you've only ever worked on websites and software that runs in a browser, so much of the heavy lifting is done by libraries and frameworks that you just don't appreciate all that heavy lifting. Some other software engineer wrote the OS for the router, for the switch, for the server, wrote the networking stack, wrote the Javascript engine, wrote the database engine, wrote the filesystem, wrote all of the caching and message queue services you make API calls to. Those people absolutely had to have deep knowledge of computer science to do what they did. Somebody had to write the compilers, assemblers, and interpreters for whatever languages you're programming in. Somebody wrote the browser itself. Somebody wrote the window manager for the browser. By the time you get to writing stuff that runs in a browser, you're so far removed from the silicon that you may as be in another world.
Not to say it's easy or uncomplicated or you deserve scorn from "real" engineers. Writing software for the browser is clearly hard given how terrible much of it is. The real problem is the incentives are all wrong. You make money by getting users addicted, which has nothing to do with the quality, efficiency, or reliability of your software. If you're writing the packet switching system for a major ISP or the SDN control plane for a million-server data center, that is all that matters.
I call myself a "developer" in precisely the way the author think of us, but I have no shame, or envy, of that. So I will first try to correct a bit of dismissiveness that I saw.
> those who took an 8-week crash course in web development
I studied 40 hours a week for 8 months to get my first job as a front-end developer. Not a full CS degree course, but there is a serious number of hours in there.
> I apologize if I’m doing a poor job of expressing my thoughts.
You are doing a poor job expressing your thoughts, because you sure look like a gatekeeper. Stop apologizing and start editing your text.
> After engineers find a job they like, they stop learning the hard stuff.
This is kind of a half-truth. I do think it is fair to say that some people, once they have a job, stop focusing on learning "hard stuff". I did it. But we don't stop learning "stuff", useful stuff for our jobs. In my case, things like CSS, UX, design, frontend frameworks.
> experimenting with a hot new web framework.
I don't experiment with "hot new web framework", I experiment with ways to build digital products in easier, faster ways. Or be able to build more complex problems.
> My goal is to articulate the importance of continuing education
It seems that your goal is to articulate the importance of continuing education of topics that you judge people should learn about.
> Over time, and often as soon as the first year of employment, “computer scientists” tend to move towards more fulfilling and well-compensated work, whether that’s well-funded open-source projects, technical leadership, or mission-critical commercial systems. “Developers” tend to continue doing what they’ve always done, learn a new framework, use an ORM to make simple database queries, or render information in a browser using a tool someone else made.
Ok, here is finally the point that I agree. You don't have to write a whole text dismissive of developers to make this point. Just focus on the argument that learning CS fundamentals makes you a stronger professional, it will reduce risks of unemployment in the long term, it will increase the chance of higher salary, it will increase the range of projects you can work on, thus increasing the chances of working a project you consider interesting and fulfilling.
Still, there are other ways to have a more anti-fragile career than learning CS fundamentals. I decided a very risky path of building projects that could earn my financial and time independence. I do think is more risky than learning CS fundamentals, but it is more suited for me.
"Over time, and often as soon as the first year of employment, “computer scientists” tend to move towards more fulfilling and well-compensated work, whether that’s well-funded open-source projects, technical leadership, or mission-critical commercial systems."
Paradoxically, the hustler who's 'learned everything themselves' has the right attitude, if not the right foundation.
The notion is hidden in the title: 'Science' is not 'Engineering' or 'Business' or 'Product' or 'Markets' or anything of the sort.
CS is like the 'basic literacy' of Software, it's foundational in some ways, but it's just a starting point.
Almost everything important will be learned after the fact.
Sometimes, a CS background means you have base knowledge you can use to apply yourself to things you haven’t solved yet. And sometimes it means you have a deep theoretical background and absolutely zero idea of how to write code; I have interviewed plenty of these folks, and they rarely get hired over someone who perhaps has a shallower CS background, but can... solve the customer’s problem, because they know how.
Theoreticians are useful, but not necessarily “better.”
And of course there are CS folks who can code; but that that isn’t all of them means it’s hard to make generalizations like this.
At some point, there are diminishing returns to increasing technical knowledge. As a former coder and now hiring manager, I value people who are good communicators and problem solvers (not just coding problems). If you want to remain an individual contributor, then sure, by all means keep focused on technical skills. But at some point, so-called "soft" skills actually become more important to advancing your career, and your career will plateau if you don't actively develop them.
So maybe what you are seeing as complacency is really about people around you applying themselves in ways other than improving technical capabilities.
On the other hand when I started my own software company, most of those skills were learned on the fly, with some awesome wins and failures along the way. You can and will be in both categories.
Most developers today building mobile apps, web front-ends/apps, or even working on the backend simply do not need foundational CS knowledge, though I suggest it would be better in most cases, and more often than not a 'positive signal' of general capabilities.
In my own experience, I started out working on embedded, designing software for fairly complex networking gear - and we were even making our own ASICS. Then moved on to a variety of other thing and I'm pretty sure I've literally never seen a sorting algorithm implemented, let along having done one myself. I've even tried to find excuses to do one, but always fell back on deference towards more established libs. My own CS background has been helpful, but more in the general sense, and I'm certain that someone with enough self discipline could have very well accomplished the same.
Specifically, none of the most applicable basic skills (i.e. language mastery, API familiarity), best practices, or even more architectural issue (i.e. tech selection), or planning approaches were derived from my CS background.
CS is a 'good indicator' and generally preferred but surprisingly not directly applicable in most scenarios. Obviously, in many cases it is.