How does one get close to mastering software engineering?
quora.com
quora.com
"Mastering Software Engineering" smells of professionalization. What's to master? Algorithms, data structures, patterns? To me, the mastery of software engineering is learning to learn and identifying what things to learn when. There will be a thing after coding, and the same kind of people will inhabit that job, I suspect.
As the saying goes, "I would sling code all day for free. It's the other bullshit that I need to be paid a lot for."
Management really should be thin layers in software companies. Instead, it is the only way to progress in the career ladder sometimes. Every manager in this thick layer wants to somehow show that they are doing something. They do this by creating more bureaucracy and busy work (but only for others).
Meetings actually counts as work for them. When they have a hundred meetings a week, they are happy and proud that they worked so much. For engineers/developers (or anyone producing the actual output), meetings are an imposition.
This is true for the sort of engineers who have no input on what is being built. Once you get to the level where you're making architecture decisions you need meetings in order to discover what to build.
My advice to any developer who dislikes meetings and sees them as an imposition is to find a way to appreciate them, or stay near the bottom of the food chain.
The meetings I am talking about are daily stand-ups, presentation for management (why don't you make some slides that we can present to the VP), the random late meeting set by a Product Manager because they wasted too much time on "other commitments" and could only get back to this one now.
> This is true for the sort of engineers who have no input on what is being built.
It's true for everyone who isn't gratuitously satisfying a desire for social dominance displays.
> Once you get to the level where you're making architecture decisions you need meetings in order to discover what to build.
Some meetings are necessary, all meetings are impositions. If you are at the level where meetings are serving your individual needs and you are calling them, you need to be very cognizant that they are still impositions, and not just for engineers. And it's good to view even your own meetings as a kind of self-imposition to be optimized and minimized in your own interest, as well as those of the other people they are imposed on.
> My advice to any developer who dislikes meetings and sees them as an imposition is to find a way to appreciate them, or stay near the bottom of the food chain.
IME, people high up in the food chain tend to also view meetings as impositions, and people who don't view meetings that way tend to get negatively noticed for it from above, below, and laterally.
> [...]
> I came into engineering because I loved to code, I stayed for the money, but when this career is long gone I'll probably still tinker with code.
Funny, this is my exact story as well. I still remember independently re-inventing the bubble sort algorithm when I was around 10, not knowing you could do better. :)
As for the topic of "mastering software engineering," I'm not sure that's possible. The stuff we learned in school about hard algorithmic problems is not what we deal with on a daily basis; or, rather, it's not what 99% of us deal with.
What I mean is that most software engineering problems are social problems. Most software only has meaning in a social context. That is, its utility and correctness are subject to social forces [0]. This is true any time more than one person is involved in developing or using a given piece of software.
In my experience, this has been true for almost every bit of code I've ever written, the exception being some fancy numerical integration code I worked on in grad school for a bit. Even then, I could probably argue that there was a social component to it, since I was building upon the work of previous grad students.
IMO, the most important skill a software engineer can have is knowing how to identify the social forces behind the software he or she is being asked to write. You aren't paid to write code; you're paid to solve problems, and, as I said, all the real problems are social. Sometimes you can do that without code, or, at least without writing additional code, and those solutions are often the best solutions to the problem.
And, once you've written code, as you said, you're leaving behind an artifact for future software engineers to see. You're communicating with the future every time you write a line of code. Since you don't know how the future is going to interpret what you've written, you should write it as clearly as possible.
It is in this way that I believe software engineering really isn't much like engineering per se, but more like writing a book. Except that it's a book with potentially hundreds of authors.
---
[0]: If you don't believe that, then, how do you explain the existence of product managers? ;-)
What’s wrong with professionalization?
To truly master software development you need to understand both, and practice both. Writing the most perfect code in the world is ace if your customer can wait. Building the most beautifully straightforward app is great if it also works properly.
You can't be a productive, useful developer if you ignore one or the other school of thought.
To be more specific, some engineers/teams are completely driven by artificial internal metrics (like ticket counts, point estimations, etc) without a concern for technical debt, or the success of the product. Instead they operate sort of like the Chinese Room thought experiment, where they translate literal requirements from upstream, but don't actually think about what that means to a user. Instead they just plug it into known concepts in the system, and wait for the bug reports to come in, then churn through those. Over time complexity builds without hope of refactoring because there is no one to advocate for simplifying the approach as it requires both internal and product compromises. Eventually a tipping point is reached where no competent engineer will touch the system and the question becomes how long business inertia will allow the paychecks to keep clearing.
And the thing here, software is simple. You take bits in, do stuff, and then its bits out. State machines there. Disks over there. Algorithms in hand. There is no real end to understanding all the details, but the game is well understood in a very reductive manner.
There is the game that we study which is making these infernal machines do useful things. Then, there is the meta game of getting humans to do things with these infernal games. These games are infinite and mastery demands you to see them for the games that they are.
In this, there is no plateau. You can operate with peak resources to explore, but the game is the game to play.
I don't agree to describe it as games.
It is just reality and life as is.
You should understand high level abstractions that run our lives or software.
But then as well you should understand that a lot of people are not having time/will/capacity to understand high level abstractions that are interesting for you.
I get angry mostly because people are not caring to understand - that is why everyone wants to build greenfield projects instead of maintaining existing ones.
It is just like person trying to open a lock with key for the first time on the old gate, they try to force it while they could wiggle the key a bit to find the correct position.
I owned couple of old cars and if you put a little bit of care and understanding they will drive perfectly well, even with check engine LED on :)
The same with team of people - don't expect that they will understand your "perfect world" just try to wiggle a bit so you can improve things enough so they work. Because what is important is getting job done and not having "prefect" world.
> Going deeper, most software people are just trying to do FAB
This is for the most part true, but I think it's better to highlight and promote the other areas being worked on.
> and most of the tools are FAB tools — there is very little CAD and even less SIM in “software engineering”.
Also true. What I see as the CAD and SIM work is being done in ecosystems, we are experimenting with various FAB toolchains, effectively each ecosystem is a CAD experiment, and the entire field is the SIM. The proliferation of programming languages exploring paradigms, source control, package distribution, build systems, etc, is moving the needle.
One thing we don't do well is identify, adopt, and refine the better ones sooner.
https://www.quora.com/How-does-one-get-as-close-as-possible-...
it is the same with engineering, there are so many subsets of it, that it is impossible to be great at all of them (eg. backend, databases, mobile dev, front-end, embdeded systems).
In order to became good, you have to became good at programing, and general algorithms and data structures, as they tend to be universal. The pick an area or two, and become good at it by building tangential things on it.
Grist for the mill:
"It’s worth noting the deep irony that the new computer tools for the engineering disciplines are almost always more comprehensive than the ones found in use by computer people for writing the programs! (There are a lot of “black screen simulated card-deck-glass-teletype” screens in use, in gross contrast with e.g. how something in EE or ME is designed and made today.)"
EDIT: Added parenthetical quote from Kay's post to explain what he considers ironic.
I think he's trying to say that programmers make more comprehensive tools for other engineering fields than the ones they use.
Fred Brooks' essay "No Silver Bullet" has a list of past breakthroughs. These are examples of applying programming to programming to ease cognitive load 10x. The essay predicts that we won't see any more of those types of improvements. But we haven't even scratched the surface when it comes to the CAD and SIM tooling for programming. There are many silver bullets left to build.
https://www.quora.com/How-does-one-get-as-close-as-possible-...
The entire comment chain is interesting to get some insight into how he's thinking about Engineering.
Can you really even master something like software development where things change dramatically almost every year?
Even though the frameworks and languages in vogue change rapidly, the patterns and logic seems to be very similar. When I have to learn a new language now it's more a matter of researching the syntax and idioms rather than learning from scratch.
I'm 32 now, have been doing this since 20, and (I think/hope) mostly kept up to date, but I'm also single/childless lol.
Also, as you move up, you inevitably end up managing/mentoring younger devs (unless you have terrible people skills or something), meaning you spend less time actually coding, this also makes it more difficult to keep up to date (in detail, you still know what's going on, but have less experience implementing).
To be fair, terrible people skills could also apply to still being single/childless at 32, I hope not...
You can somewhat mitigate this if you want to still keep your hands in the code by finding the intersections of two or more tech stacks that take two or more others to even approach. The inevitable impedance mismatch between say two stacks means that one person familiar with both is usually far more effective than two people who have to feel their way through the mismatch.
As a senior in this position, you often get to do the "fun" parts of throwing together a proof of concept, validate your mental model of the impedance mismatch, outline the details to iron out (usually known-complexity work like documenting, hooking up to various plumbing for unit testing, tracing, logging, monitoring, internationalization, etc.), and go on to the next fire to help put out.
I truly can't understand thinking this. I've worked in software engineering for over 10 years now, at a wide range of companies from tiny startups to large tech cos. We easily make 2 - 5x more than the median wage, work indoors in a comfortable environment sitting (or standing, for anybody who prefers) all day with high end ergonomic equipment, often get unlimited vacation time, can often choose not to work more than 8 hours a day if we don't want to, and all while doing something that many of us got into simply because it was a fun and engaging thing to do. And just in the past year millions of software engineers have gotten the option of doing this fully remotely, freeing them up to live anywhere they want and never have to commute. What a time to be alive!
Comfort doesn't have a lot to do with burnout or job satisfaction unless you're coming from a blue collar mostly manual job.
> many of us got into simply because it was a fun and engaging thing to do
That's one of the problems. For a lot of folks, it was exactly that 'was a fun and engaging thing to do' and then you get do it in the context of office politics, tight deadlines, need to sacrifice professional ethics and ship something half-assed because that's what the business needs, etc.. These things, repeatedly chip away at your happiness. Would you be more unhappy without the ergonomic equipment and all of that? Most likely yes. But ping-pong tables and the unlimited vacation (which is at the discretion of your manager anyway) doesn't always compensate for shit work or the nervous breakdown from having to sit on an incident bridge for 4 hours...
I need to admit, I love my job and I love this profession. But I do understand when people say it's draining.
Oncall is somewhat unique, and that can be tough, but if it's particularly rough and there is nothing you can do to improve it that's a good sign to find another company (or perhaps another specialization within programming) because it doesn't need to be that way. Incident response is mostly a self-imposed stress, and I've found it to be something that you can learn to manage better and not let it affect you so much, by being prepared with good playbooks, as well as accepting that it's not the end of the world and that all you can do is try your best.
I'm not denying that life itself can be draining. Working any type of office job can be draining. I just don't see how programming is particularly more draining than all the other jobs people do.
Sure, some devs become managers but there are only so many managerial jobs. Some also drop out but not in vast numbers. From my own experience, I see quite a few of devs my age (40) and older. For the most part we're senior engineers, some are managers. I don't know anyone who's left the profession. Just my anecdotal evidence but I do feel this phenomenon is overstated.
I'm a Scala developer working on fairly typical, distributed systems / web apps. I have no plans to stop Scala since I love it but any JVM language would suit me. I'm also into system design / architecture in the distributed system, "reactive" architecture domain. Those are the systems I tend to work on, massive web apps consisting of many microservices.
I'm always learning and trying to be on the edge of my comfort zone which is key I think. If I'm not messing with Scala then I'll be learning more about Kafka, Docker, SQL / Mongo, Terraform, AWS and a bit of Linux.
I'm far from a master in any of these things and I don't think I'll master many which is fine. I know enough to do my job and I keep on learning but the breadth of what I have to know is too broad to master it all. My true passion is programming, particularly functional, and I would love to master that for myself as much as anything. I guess you have to chose where to go deep and where to go for breadth. You can't master it all!
Scala is an absolutely awesome language, one that I love. I've had some of the best times building distributed systems with Scala + Akka and frankly would have loved to lean into that ecosystem and become a master of it.
Unfortunately, the market for that specialty is relatively small when compared to the broader software engineering market. I worried about specializing in a language that was niche. Compare it to, for example, the need for React, React Native, or even simply Java.
I have chosen to be very flexible in my career and learn many languages, across many domains. This, I think, is the source of some of my feelings of burnout in my 30s. It's just a never ending grind. When I do find a domain that I really love, like firmware, or Scala... I'm forced with the difficult decision of depth v.s. breadth and breadth always seems like the "safe" route, especially if you're willing to jump between frontend or backend and whatever is the flavor of the week as far as frameworks go.
I've heard horror stories of engineers who dug into their tech stack and never really kept pace with the industry, only to find themselves disposable, and inflexible, in their 40s and 50s. Whether or not those anecdotes are worth worrying about is certainly up for debate. I tend to be a worrier so, it's no surprise I've chosen the safe route.
As for specific languages, as long as you're aware of a couple of paradigms, eg OO and FP, I figure it's not that hard to switch. Once you've mastered one or two languages you can generally be productive in another relatively quickly. We don't necessarily hire Scala devs for example, just engineers we think we can train up.
I'm lucky in that the Scala job market is hot in London but I'm fully prepared for that to change one day. I'd probably target Kotlin or Java jobs if that happened but also open to anything interesting job-wise that will have me. I have my tech preferences but if I need to switch, and someone will allow me to learn on the job, then fine!
I've learned 7 programming languages, COUNTLESS frameworks (both backend, and frontend), across many different domains... frontend web, native mobile, hybrid mobile, backend, even dabbled in firmware for a few years. All of these in a professional context, where I was shipping real live code.
If there's been one constant in my career it's been change. Change is constant. I can absolutely expect that in a given ~3 year timespan, I will need to learn a new language, domain, framework, etc.
The result is absolutely that I feel like a jack of all trades, but a master of none. I can pick up new languages and frameworks quickly, but I don't ever feel intimately familiar with any given language or framework or domain. I don't feel like a "master" of anything other than, perhaps, mastering the ability to learn new . It sucks.
Perhaps not coincidentally, I've struggled with feelings of burnout. Looking back on my career over the last 10 years, it feels like a constant sprint. Of course I'm a better developer today than I was 10 years ago, but I know that the future only brings one thing: New technologies, new frameworks, new languages. I will need to learn those, and only a fraction of my current knowledge will apply.
It's hard not to feel like you're treading water in this industry. I can learn anything, but for long? For how many years, or decades, am I willing to do this? It feels like I need to either accept that this is the reality of this industry, that the biggest skill I can have is willingness and ability to learn, or, accept defeat.
Jokes aside, the grind and churn is real. Just try to find a job without the fad chasing.
after some point there's not much one needs to learn to be productive.
I feel no pressure not knowing vue, django, fp-ts, rxjs, Rust, Zig, C++20. Rarely there's something you can do with new stuff that you can't do with old.
I think you probably meant "than"... although that may be profound the way you spelled it.
Most of my team are 40+, we're doing fine.
> Can you really even master something like software development where things change dramatically almost every year?
This is a good point but things don't change that fast.
As others have mentioned, the further I too progress in a technical career, the more I realise how little I truly understand in the grand scheme of things.
That, more than any other description, has always resonated with me. Any beginner can create a bird's nest of code that is too complex for even the most experienced developers to fully understand.
I've been at this for 40+ years now, and if I'm struggling to write something, it's usually not because I don't know how to implement it. It's because I'm trying to pick an implementation that will be simple to understand, verify, and use.
Second, what is "mastering"? It's not being able to crank out leetcode problems. It's much more being able to see the upsides and downsides of various architectural, design, and implementation decisions, and to not make the wrong ones.
By doing it, doing it a lot
And then listening to, and watching your users use your stuff. They'll teach you more about creating better software than anything else.
The rest comes naturally.