Ask HN: What does mastery look like in software engineering?
When you think of someone who has mastered the discipline of Software Engineering what comes to your mind?
Masters in my organization have an extremely high ratio of value shipped to hours worked.
This means two things. They are able to discern and avoid work that does not provide value (this is often their greatest skill), and the best ones can direct a whole team away from large swaths of work that is not valuable. And they are able to build systems such that the cost of adding more value to the system remains constant as the system grows in complexity.
Can't overstate how valuable this is. A team which can expose its members to this kind of leadership by example and direct collaboration will grow and outproduce teams which don't by an order of magnitude.
junior dev here...could elaborate a little on what this means? Generally speaking the PM is the one creating a roadmap and informing what is the highest value item to work on. I must be misunderstanding what you're referring too.
A Senior Engineer knows what the highest-value work is and is influencing/driving the roadmap. A Senior Engineer says 'no' more often than 'yes' and backs it up with a 'why.'
Most Seniors are driving the roadmap simply by virtue of being a Senior, and most Juniors have no opportunity to influence the roadmap because Seniors control the decisions.
Of course ideally everyone can contribute, but I think that's relatively rare.
I how ever could already experience a more open scrum process in which we all have a say.
For me it all started in an university project with four (incl me) people. I denied a good portion of the requests my supervisor gave me there, not because they were bad, but because they simply did not fit into the time budget or were not realistic. She was actually very happy about that because she was not that tech savy back then and learned a lot threw the process. Very good experience for me and we finished the project on time and exceeded expectations.
A problem came up. Many of the libraries were government funded, and required physical possession of any books they purchased. I was in a meeting with the c-level execs, sales directors, and pms, and they were planning out the cost of building an organization that would produce a cd-rom version of the textbook products. They were going to need a large team to manage creating the books on cd, and for managing the production and mailing of the CDs when orders were received. It was going to cost a fortune.
I told them to hang on a minute: I'm already converting the xml for these books into html for display in the app. I can easily generate static html too, write it to disk, and generate an iso file, every time we publish a new book. Then I can add a download link in the webapp, and if they need a physical copy they can download and burn it to disk themselves. We'll include instructions. This will take me maybe an extra week, and we'll have no ongoing operational costs.
I think I got a $50 Amazon gift card for that suggestion. Or maybe a Starbucks card.
I am hoping this will be an area I can eventually excel in. I was a senior PM for a while and decided I wanted to write code instead. Been going down this crazy career change for about 6 months now. Right now I am just trying to survive and learn the basics of software development.
When I was PM I certainly had the luxury of working with invaluable devs I could brainstorm with on ways to optimize a feature idea that maintained customer value in the shortest time possible. I need to learn how to become that invaluable dev I had.
(I wish you had gotten more thoroughly compensated for saving them a fortune.)
We are called Software Engineers, but what we really are is people who solve problems with code.
The important part is solving the problem. The fact we do it through the medium of code isn't that noteworthy.
If solving the problem requires knowledge from outside the domain of programming (Spoiler: it almost always does), then you learn what you need to from the relevant domain.
Experience flips this equation around. The PM is creating the roadmap based on my input and expertise to decide what will be the most valuable use of resources. You move from construction worker to architect.
It may also depend how much you're micromanaged vs. picking your own path. Sounds like you're saying none of your work is self directed, and you don't participate in design decisions yet?
Anyway, even if you don't have much say there's still ways to direct your work. You can push back against whatever is demanding the low value work.
Unfortunately in a large software engineering organization tact and political savvy is more important than raw engineering skill. Once you've been in the industry long enough, you'll understand it.
One of the problems that's common in organizations is that most of the work at the junior level is highly focused on individual contribution and that completely flips at some point in your career.
I tend to encourage folks who come to me about quitting to explore opportunities within the team/organization/company before leaving so they can leverage all the social capital they've built up.
The more important question is "how do you cultivate mastery?" How does your organization get to a place where people can develop mastery instead of the organization trying to create it instantly by hiring "the right people"? Regardless of what people may claim, hiring is not a solved problem. It's _really_ hard to identify candidates that are true masters at what they do (let alone getting them to quit their job and join your team).
By the same token, it's also _really_ hard for true masters to find a place that understands their value and, more importantly, will allow them to do their thing and provide career satisfaction. A lot of talented people are locked out of jobs because someone thought their career trajectory wasn't quite right/impressive.
Training and mentorship, however, are better understood and have lower stakes. It makes me think of Bell Labs. I know a couple folks who worked there who had very "unimpressive" backgrounds, they started as techs but rubbed elbows with stellar talent in a place that cultivated discovery and collaboration. Over time while there and in other places they were able to rise to principal design engineers, beyond what many who went to elite universities achieve. They credit their formative years in Bell labs with getting them started on their career path.
The thing is, Bell Labs didn't have an epically selective hiring process, nor was the compensation insanely high. They provided an environment that _attracted_ people with mastery and those who were passionate about what they do, they developed talent within their walls rather than trying to merely buy it or find it.
This was for electrical engineers, but the same ideas can apply to software engineers.
The importance of good politics cannot be understated. Politics in this case is roughly defined as getting people onboard with your plans.
I say this because as a finance guy, I've been on lots of projects that look great on a spreadsheet and marketing, sales, or somebody else is really pushing for the investment. That's the point where a technical person takes over and builds what's asked, or a really great technical person may ask a few questions and raise a few points nobody else considered and the whole bubble bursts as we realize it's not feasible/or we can test it in a certain way before investing big time/money into it.
Probably not mastery of SE as OP asked. But mastery as a business partner within an organization.
Either the engagement with technology doesn't happen early enough, or when it does, the technology point of view isn't given the weight it deserves. The ideas have already caught and too many people are invested and have 'faith' it will all work out.
The crux of this is that many of those people who carry the weight in decision making don't actually carry any risk or responsibility for delivery. Given the scale of engineering and time to deliver, by the time it comes to fruition they've moved on, so they never experience accountability for ignoring the advice.
For instance, in my experience at a FAANG, the issue was often that a new technical solution was proposed, architected and built before any business/market people got involved with the result that loads of amazing, brilliantly architected and completely useless projects got built.
(My favourite time was when a (really smart, to be fair) software engineer rediscovered the normal distribution while looking for a way to reduce storage requirements for an analytics product).
Again as a finance guy, this is a major skill I have attempted to develop for myself/team. When projects spin up, I’m typically at forefront and involved in every project technical or not (controlling the purse strings). Many finance guys are know it all’s TBH. They build their ROI models, etc. and making dozens of assumptions it’s then treated as gospel. For some reason, it’s valid to ask “why is this project underperforming our expectations” but it’s often taboo to ask “did our expectations even make sense”. I try to change that where it exists too.
All to say, people like me find people at the onset of the project to gather inputs. If I know you as well versed with track record of valuable insights, I’m more likely to reach out early. If I have worked with you at this stage and you your input is just one of “we need specs/tell us what to build and we can build it” you’ve not added any value to the conversation and I’m likely to not include you next time because I know I just need to bring you specs once project is approved and that’s a different conversation for another day.
This is why I say “business partner”. Being able to understand your company, the politics within, how decisions are made, where you can inject value along the way. These are all traits of people with fast career growth in slower growing firms. You transform from Programmer #13 to people knowing and respecting your name. This is important in talent reviews, etc.
I find some in finance and IT get along fairly well and can build a bigger rapport fairly easy. We’re all excel nerds and many of us have taught ourselves how to program to an extent (VBA, etc).
Depending on the organization, we tend to be more focused on leadership support. A junior analyst will get much more face time with VP and up than down. So that’s part of the problem is those connections just don’t always exist. When they do, it might be with an IT director which has maybe become too far removed from codebase or implementation challenges. It happens with every department within the organization, not just IT. My experience is outside the software industry so may work a little different if sole function of the company is software.
Getting the users in the org to, you know, actually use that well-specced, well-built project, which was delivered within time and budget (or with only a small overrun). I mean, use it at all, or use it enough for it to deliver its benefits. Real life issue. Seen it on at least a few projects I've worked on, including two early in my career. Inertia, luddite fears, turf wars can be some of the causes.
Watch "The Effective Engineer | Edmond Lau | Talks at Google" on YouTube https://youtu.be/BnIz7H5ruy0 - at about 3:21 in.
It's usually helpful to have an internal locus of control, and look for things in oneself (here, soft skills) that can be changed. However, sometimes reality just doesn't afford opportunities in a given situation. Accepting where that is that case means being able to extract oneself from a situation rather than wasting time on it.
An organisation that does this is broken.
Most are.
While it is important to realise that this brokenness is in fact widespread, I don't think it should be normalised.
In your analogy if I get really good at getting stuff passed through city council, what would I optimize for ?
A: Building better bridges ?
Or B: building better money making bridges (potentially of of inferior quality) ?
I'd bet there are more takers for B. And at that point we now have better city council optimizers than better bridge builders.
Lets say we now make a team of some folks from both groups. Now who's going to able to get more people onboard their plans? Its the city council optimizers again. And now some bridge builders figure it out and become better optimizers than builders.
We then take it a step further and add competing teams. Same thing repeats. The team better at city council optimizing beat the better builders all the time.
In this whole process its the quality of the bridges that stays the same ( in most cases it goes down) and the costs of it keep going up.
My sad, unfortunate observations in the field.
Love to see a way out.
Edit: in this way of thinking , the assumption is that there are higher chances of making more money from poorer quality bridge than a more advanced, higher quality bridge. One exception to that is the strategy where you first provide better bridges at lower cost and then bump up the prices for maintenance.
On the long run, if you stay in such a company your market value will by reduced as it is
(a) psychological very difficult to resist such culture to influence your mind, unlimitedly transforming you.
(b) will eventually turn your life into a saga of bitterness and resentment, lower self esteem.
(c) arguably, such a company will ultimately fail in their business.
so you're probably better off take a possible temporary pay reduction for a better happiness and higher pay on the long run.
For instance, having an easy build process, or understanding version numbers, or preventing an upgrade to this year's compiler.
I feel it's a core part of my job to keep the rest of the eng team happy and producing value. A lot of times this comes down to seeing problems as they arise (ideally before them, but it gets harder to convince management of future issues if they haven't manifested yet).
that's a tough one, maybe the tough one.
It's hard to prioritize "trivialities" over features, especially when you haven't even given thought to defending the obvious.
Thankfully some of these things are being codified somewhere externally, say:
https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
Typing hardly registers as a factor when it comes to how long it takes for me to code anything, at least.
I can. Actually, when I learned touch typing I realized how much was I slowed down by typing before.
You might argue that I need to think for a time before I start typing. That's right. But:
1) Once I start, it flows and it's like I'm reviewing the code as I type, correcting and expanding.
2) Writing fast somehow makes me think faster for the next "pre-typing" thinking cycle.
If I'm solving a hard problem (i.e. architecting a system or coming up with an algorithm) then 99% of my time is spent thinking rather than typing.
-- Do I bother you?
-- Don't waste yourself.
-- What's your style?
-- My style? You can call it the art of fighting without fighting.
-- The art of fighting without fighting? Show me some of it.
Leadership happens from the front and doesn't really have anything at all to do with year end reports and making sure everyone shows up on time. Management rarely does more than HR busywork. The way many engineering mangers work they often have skills so outdated as to be effectively non-technical.
Leadership can happen at all levels regardless of title. Leaders often are some of the most skilled technicians, able to have an outsized impact on both the team and the direction.
Is it an official title? I keep seeing it everywhere now, everyone is a "leader." What exactly does it mean? Do you proclaim yourself a leader, is there some vote on it?
From my perspective it's completely meaningless and usually ego stroking.
Given that, predicting which projects will produce that value, and which designs will accommodate future additions of value, is engineering.
I still like Parnas's On the Criteria to Be Used in Decomposing Systems into Modules, where the choice of "information hiding" requires predicting what is likely to change - and predicting value.
Novice engineers will struggle to get something done. Intermediate engineers can get things done but fail to see the tradeoffs. Expert engineers can identify possible solutions with different approaches.
- Working with experts
- Writing and deleting lots of code
- Participating in code reviews
- Reading even more code from others
- Watch & read technical presentations
You can never truly be perfect, you can only approach it. Understanding your limitations is just as important as understanding your capabilities. Depth vs breadth is hard and you only have so much life to go so deep into so many domains.
Attending retrospectives after things go wrong is a good way for experts to introduce their thought processes to less experienced engineers.
Asking "why?" when senior engineers deliver feedback during design review can help you learn that a trade-off existed when you did not see it.
Only, I didn’t like having two to chose from. Something told me three would be better. So we paired, I wrote mine, then hers, then just made another one up on the spot. Once we agreed they were complete and implementable, I shopped them to the team.
About 2/3rds preferred the made up one. Including me. So that’s what I implemented.
An intelligent man learns from his own mistakes. A wise man learns from the mistakes of others.
In more detail, the steps go like this.
Find as many solutions to a given problem as you can. Bad engineers run with the first solution that comes to mind, letting confirmation bias drive them.
Evaluate each solution for its costs and benefits. Imagine two steps in the future when the solution is implemented. What pains are there?
Search for creative new solutions that create win-win scenarios. That's riding the solution curve.
Given all viable scenarios, compare the costs and benefits against the quality measures for your specific context. Some projects value speed over precision. Some projects value performance over extensibility. Some solutions are easier to change later than others. This is choosing a specific point on the solution curve that best fits your context.
You can apply that method of problem solving to any problem, large or small. You don't need a ton of experience to practice it.
I’ve repeated it that exercise with junior engineers to great effect. Some catch on over time and start intuitively considering the trade offs of a few reasonable solutions to a problem; some don’t.
I never reflected on what he was doing there; thanks.
The best software engineers know when to solve a problem without using any code at all, like the classic "just do it manually" for a complicated task that is worth more than $x per task.
- Identify and document possible solutions. Think hard about why each solution is good and each decision is bad.
- Within each solution, attempt to find common patterns in that problemspace (when working with Sharded databases, this approach brings these results, etc)
- Try to match your personal and organization requirements against the various patterns you find and the pros/cons you defined previously
During implementation
- Keep a running list of things that seem weird, or things that seem great, or things that turned out to be untrue
- Don't stop implementing, but for each thing you found that wasn't great in step 1, try to find a better way to approach that thing (while moving forward)
Post implementation
- Compile all the notes you made in pre/during into a manageable list of goods/bads.
- Use this to drive A.) iterations of your solution, and B.) future solutions
If you do this enough times, you'll have a pretty decent list of your experiences in a space, and you'll start to notice patterns as you try different things and become exposed to problems, their solutions and their tradeoffs. Eventually you won't even have to look at your previous notes very often, as you'll have built up a pretty decent amount of experience in various problemspaces such that you can predict what goes well/doesn't go well.
Also worth noting, this type of behavior doesn't really have to be applied to software, but will also gain you big points with future employers when interviewing. You'd be surprised to find how many candidates never stopped to reflect on the work they'd done, why it was sub-optimal, or how it could be corrected moving forward. Response like "We did it ____ way because that's how we always did it." Which, from a progress standpoint, is basically a none-answer.
Note: this is obviously just my opinion, and I'm no expert in "Becoming a Master of Stuff", but these are the methods that I have used and seen my peers use (in one form or another) over quite a long time. Sometimes not as obvious (doing these things in your head vs. writing them down), but the shape is always pretty similar. Ultimately I think being reflective is one of the best skills a person can possess (with respect to employment and I guess also relationships). Acting without thinking is reckless, I think.
As they say, pain + reflection = progress. If you don't wanna reflect, good luck enjoying that pain without progress.
Naturalistic Decision Making - https://en.wikipedia.org/wiki/Naturalistic_decision-making
Expert engineers have the ability to think about a problem in a broader context than the engineering problem at hand.
- Howard Aiken
Now, you might be thinking "that's not true, if I implement a function to add two numbers, and it returns 1+1=3, that's wrong. So that's a clearly wrong solution." But I argue: is it wrong? What's your context? What are your acceptable trade-offs? In many real-world applications, precise correctness isn't always critical. Sometimes, sacrificing accuracy might be acceptable, and may save on cost and/or complexity. Other times, that's not acceptable. But it's worth considering even these types of non-obvious trade-offs, not just the obvious ones. That's what an expert engineer does: ask the right questions and identify the right trade-offs, and they've become efficient in that process from experience.
An expert sees a problem space as a wide range of trade-off dimensions, including:
* computational performance
* memory performance
* implementation complexity
* maintenance complexity (understandability)
* accuracy
* reusability
* design time
* reliability
* debuggability
* bug-proneness
* scalability
* etc.Just like a master woodworker would consistently and reliably deliver good furniture.
Or a master author would consistently and reliably deliver good books.
The key points are:
* Consistent - It wasn't just a fluke or luck, you know what you're doing and can repeat it.
* Reliable - You know how much effort it will take and can deliver it.
* Good - It's not crap, people want it.
Mastery would imply that you can do it all, and do it right, on any project, and there is just too much change and evolution for that to ever be true.
My goal is simply that I continue to improve.
I also have trouble believing you've been coding for so long and can't take pride in any of the code you've previously written.
I think I do what I do pretty well. The folks I work with seem to like it.
For example: I know people who have been skateboarding for decades and are just barely OK at it. How you practice matters.
I think there is mastery, but only in knowing that there is no mastery.
"I seem, then, in just this little thing to be wiser than this man at any rate, that what I do not know I do not think I know either." [Apology of Socrates, Henry Cary literal translation]
I take it one step further, I don't believe mastery exists, or rather we don't have a consistency on what mastery 'is'.
I prefer the saying something along the lines of "expertise* just means you get in more trouble when you do it wrong"
* Expertise is replaced with 'black belt' for martial art expertise
I play violin, and know and admire many masters. What makes them so good is fun to study and pursue because they are worth emulation.
But it's not some secret or hack. It's a million small things and a lot of feel.
When we interview people, we do so in such a way that we believe someone is capable of showing mastery of the field, where those who have been in the field a long time and agree with your philosophy here know that about all we can can hope to achieve is humility.
It also feels like this fundamental to why this field has not been able to mature to a point of being a profession in the same form as other fields - because while we can teach a lawyer the law and we can teach a doctor a particular way to perform a technique, software is all just a bit wet and sticky.
I've both seen and written code like that, so it does exist. Like anyone one else I've also written code that causes future problems. I believe I'm at a point in my career (25 years in) where I now write more of the former than the latter.
Doesn't matter how much time/effort I expend, there is just no way to keep up to date with every moving piece in our industry (from hardware, to firmware, to software, to laws, to programming languages, to cryptography/security, to web standards, to multimedia standards/codecs, to physics etc). It seems like every single action that we take comes with trade-offs and that we can never make an ultimate good choice - that choice is only valid for a short time before it's trade-off gets triggered.
A carpenter has a set of tools. He has a set of materials. He has a set of blueprints/techniques. He can consistently build table with x qualities. When he is done with the table, it is done. He doesn't have to spend all of his time relearning how to build tables over and over, and for old tables to quickly self-destruct if he stops baby-sitting them. Or for some 12 year old carpenter in Russia to cause his table's wood to rot.
Every year that passes by, humans accumulate more and more knowledge. It would be impossible for a few individuals to know everything as their time in this life is limited. Thus we get forced to specialize, else we can get pulled into the general knowledge stream which can be overwhelming.
To be exact: "The developers came back and said it would take two months and exceed their budget.
Rather than having a big confrontation over the issue, I told them to just send the project to me and I would do it myself. Cass Everitt had been doing some personal work on the iPhone, so he helped me get everything set up for local iPhone development here, which is a lot more tortuous than you would expect from an Apple product. As usual, my off the cuff estimate of "Two days!" was optimistic, but I did get it done in four, and the game is definitely more pleasant at 8x the frame rate.
And I had fun doing it."
I guess that for most of the guys, if they had opportunity to work on something with Carmack or Id Software in those days, it would be a priority to show yourself in best light and even work for free or heavily discounted price if needed to secure future deals. Especially since all source code was provided and basically the whole thing was just somewhat simple porting engagement.
Software engineering, gives you some degrees of freedom, therefore it's kind of an art form. Therefore mastery is about intent and the ability to realize it.
In this canvas, you can ask yourself, is the bearer wielding the tool or is it the other way around.
For me a proficient tool user is no master until he makes his tools his own. He can become one either by creating them, or by understanding them deeply enough that he could recreate them.
There are then various levels of mastery and mastery over various sizes of domain. Levels are more about the capacity of imagining new tools, and having new visions in order to advance your field in some interesting way. Growing domain size is about trying to have bigger and bigger tools while still retaining control over them.
Thanks for the links.
A long time ago I saw a good talk by Jonathan Blow wherein he made a point that really resonated with me. The point was that to get good at programming you need the experience of shipping a lot of programs and thus you should optimize your career around shipping as many good programs as you can. Contrast this with the common advice of being an expert in databases, or distributed systems, or OOP, or testing, or something like that...
Edit: found the talk: https://www.youtube.com/watch?v=JjDsP5n2kSM
Clean, well-documented projects showcasing demos that end up having huge impact. Some of these projects single-handedly spawned entire tech ecosystems. (see FFMPEG, QEMU, JSLinux)
It's one "direction" of mastery. Software invention, at grand master level.
* Consistent delivery of quality work
* Instead of complaining about bad requirements that threaten system integrity, happily works with stakeholders to compromise on them accordingly
* Can consistently describe every detail of every bit of work they have done, and answer any curveball questions about it too
* Succinct and to the point, doesn’t bog down meetings with tangential arguments and what-aboutisms
Could probably keep going but that’s good for now haha. I know I don’t even satisfy all of these points myself at all times, but I certainly try to!
Mastering the language but understanding why the language was developed what problems it looked to address.
Reducing the cognitive load of the design using the language.
Understanding the fundamentals of the types of applications you will be building. SCRUD Architecture Apps Streaming Architecture Apps and Games which are usually a combination of both.
Software Systems from first principles are read and write systems. How do you represent that as a program and reduce the complexity?
How you view software engineering ? and explain it to others what is the method of representation you use to encode software as plumbing? is it text based? time flow graph node based? stack based ? binary based ?
Software engineering as a team sport, how do you get a bunch of developers to reduce the cognitive load manage the relationships and data in the code and how do you measure that like a science.
I am also sure there is a lot of valid advice here as well. This is how. I see it from a mathematical / science perspective that pushes pragmatic narrative.
Many of these traits are fundamental, universal, and even philosophical, than is general perceived, and transcend technologies and stacks, and to a large extent, time.
And yet, I think, the above traits can be described as good, and not really mastery in its truest sense, like in other disciplines like music or painting. Software engineering is way too diverse, fragmented, chaotic, and ever changing.
* I think an intermediate level of mastery is when you realize that a computer is a logic machine built to serve humans and it will always listen to you if you're willing to put in the effort. You can always go a level deeper. Great engineers know all of the tools at their disposal and will bend the computer to their will if they need to. At the same time, they know the difference between a great low-level solution and a shoddy hack.
* Our industry is one of the few where we spend the majority of our time fixing our own mistakes. We call these mistakes "bugs" but they don't just appear, we produce them. The best engineers produce fewer bugs than they fix and rarely end up in a "whack a mole" situation where fixing one thing breaks another thing.I've worked with some people who definitely seemed like "masters", and this was true of all of them. Quiet patience and an unshakeable belief that this thing is going to be fixable, even after trying multiple things that have gone nowhere. And they're almost always proved right.
It also made me realise I need to get out of this business as soon as possible, because I'm the polar opposite. After a few failed attempts my brain goes into panic mode and the problem seems insurmountable. Then I "wake up" an hour later after a bout of procrastination, no closer to a solution.
I often wonder about the chicken and egg of how people attain this state. Are they first and foremost technically brilliant, and the confidence emerges from that? Or is it more of a personality type that has allowed them to solve lots of problems and become technically brilliant along the way?
1. I’ve run into the same or similar problem before.
2. I wrote the code that does this thing.
In case of a time pressure, I just focus on workarounds instead of analysis. Once the problem is no longer burning, it's up to the project priorities to investigate the root cause.
I also learned that panic never solved any of my problems, so when I notice that I am moving into panic mode, I stop for a minute and cool down. This is a skill that can be learned.
When you have a high level of comfort with all of these things when you see a problem you can often guess *where* the bug is and that's half the battle right there.
Have you ever noticed that your friends will ask you to fix their WiFi or an app on their phone and even though you have no real expertise you can normally do it? That's a version of the same thing. You understand the general principles of the system and you're able to guess the general location of the problem and start flipping switches until it goes away.
You can't master Software Engineering without mastering interfacing with the rest of the organisation.
I don't care how intellectually pure your implementation is or how fast you deliver things. If it can't easily connect to, use and get used by the rest of the business, it's a folly.
A master of software engineering delivers what the business needs in the minimum time possible.
Some businesses care about maintenance. Some don't.
Some care about UI/UX, others with a trapped audience don't.
Some care about efficiency and optimisation, others don't.
You can't be a "master" until you have a sense of what your business is interested in. Not what it CLAIMS to be interested in, but really cares about.
It's been different in every company I've ever worked for.
Good luck.
Yes, that's right. It's also a choice. You don't have to work for a company who does not care about maintenance.
At least I wouldn't, people should have own their standards.
With that codebase you are going to spend your 1/3 of your days. It really matters.
"Master" really is a great word for it, you have complete faith in the individuals word.
I would trust a master to take the appropriate amount of time to do a research spike or to bring in the right people in and understand how to get going.
The other two I thought of are Peter Norton and Linus Torvalds.
Fabrice, despite his tremendous contributions to the field, doesn't reach for acclaim and just seems to go his own way.
Peter Norton was a genius in his field. His book on Assembly Language was one of my favorites as a youth. Norton tools were impressive for their time.
Linus, as the namesake for Linux also made a huge impact on the field... and comes off as somewhat of an uncontrolled lunatic.
I don't though think that these three people are the highest level masters in the field. I think they are just the ones who have chosen to do things that have high impact and visibility.
Ha! Would you consider Musk also as a Lunatic? :-)
What? Linus is one of the most Rational, Direct, No-Nonsense, Get Things Done Leader there is. Whenever i hear some self-appointed "expert" pontificate on Leadership/Management/etc. i always think of what Linus would say/do in that situation. People only think of him as a "Technical Guy" but his Leadership/Management of something so vital, so huge, so distributed, with so many "employees" and with so few resources, so many stakeholders is what is amazing to me. Nobody comes close.
You really need to take back the "uncontrolled lunatic" phrase.
I'll agree that many of the things he says are reasonable. An example is his recent defense of long lines and dismissal of the insane "80 characters per line" peeps.
What I mean specifically by "uncontrolled lunatic" is that he often ignores convention and current opinion and does and says things that get him into trouble. He also has a temper that he often is bad at controlling. I myself have this issue, and I don't see how one could say he does not have issues in this regard.
While it requires some amount of perpendicular thinking to be a master of software, having such an attitude and working with other people in that way will cause too much conflict that is unnecessary.
I think what you are missing in my statement is that I consider calling someone an "uncontrolled lunatic" as a sort of compliment.
I agree also that he is less of a technical person and more of a manager. His skills lie in herding.
On the other hand; Git is a flaming piece of garbage and he is directly responsible for its popularity... so... He did though reasonably tell people he didn't intend for anyone outside of kernel developers to use it. I just wish he corrected its deficiencies. Maybe someday he can apply his "arbitrary character restrictions are silly" logic to Git comments and become less contradictory.
Genius cannot be constrained by every single artificial social more and etiquette; we are Humans not Automatons. As long as they do not lead to destruction of product/project/self/others it is merely a character trait which makes the person unique. All successful companies know this and "manage" their most important people accordingly.
Justifying treating others poorly just enables more people being hurt. We should not reward people for this sort of behavior. Turning a blind eye to it because "they are great software engineers" is not right.
Your comment treads very much into the "for the greater good" mentality where you can justify anything based on the results.
"Unique character trait" my ass. Obnoxious aholes are just that. Obnoxious aholes. They need to fucking stop and be more kind.
It is also quite ironic that while you talk about "being nice/kind" your posts are the ones containing inflammatory phrases like "uncontrolled lunatic", "arrogance and obnoxious attitude", "my ass" and "obnoxious aholes" !
Coming back specifically to Linus, he has never been any of the above. He has been direct and harsh when needed (there is a world of difference between this and the phrases you have used), otherwise he has shown a lot of restraint. For example, he has said that due to the nature of email communication, one has to be direct since a lot of non-verbal and personal nuances cannot be conveyed which is exactly right.
I am sure we can all recollect email chains/discussions/meetings where nothing was ever achieved because people did not want to offend somebody with thin skin in spite of the fact that they were squarely to blame. Being direct and calling them out is not being obnoxious/arrogant but simply moving things along towards a goal. To be nice/kind does not mean everybody has to be mollycoddled and tip-toed around due not wishing to "hurt their feelings". Engineering is a hard science and while we have to follow social mores and etiquettes this should be regulated (i.e. we should know when to break them) so that progress towards the end goal is not derailed.
Linus has said many attacking things publicly under a position of authority.
What I say here bears no weight because if what I say is disliked I am simply downvoted and dismissed.
I disagree that being offensive solves anything. I am fine ( as you've pointed out ) with being extremely offensive and attacking the shit out of anyone I see fit, but I don't think it is particularly effective or a quality trait.
Anytime you say things in an offensive way, you are intentionally being hurtful. This is not a good thing, and I don't think is justifiable.
My doing it in no way makes it right, nor invalidates my judgement of those who make similar statements to a broad audience.
Consider the effect of me ranting here. No one is going to go: "Wow I'd better pay attention to what permille42 says he is an expert. He must be attacking people for a good reason."
Nope. Anytime I rant most readers will just immediately dismiss everything I said, mostly because my tone tends to be so aggressive. I don't think such people are wrong to dismiss as such aggressive statements do tend to be from people who are offbase and don't know what they are talking about...
I think of Linus, Lars, Anders, TJ, Rich, etc
This article from 2014 highlights just how many libraries he has written and how good he as at writing them: https://medium.com/@kelas/how-is-tj-holowaychuk-so-insanely-...
funny to see how this is still relevant.
i'm sorry to say, but anyone who read this ancient piece at least marginally past tl;dr would know that your conclusion can't be more false (must be due to my poor command of your language, sorry about that.)
i originally wrote this text in response to eponymous question on quora, which saw some unexpected gargantuan success with 100k+ views in the first year or something like that.
tj said "meow" under that quora answer, and disappeared for good - while i was left exposed to some really agitated tech bloggers who wanted an interview about how did i get so smart to figure out that the entire headless chicken called node community was just brutally duped.
no such interviews could be granted, but in wake of the story some people bothered to look for the clues in his gh activity - mostly frequency analysis of "his" commit logs in a galaxy of repos. this is why most tj's repos were promptly republished elsewhere after that.
bottomline: the article says that tj was a SCAM, so there's that.
This answers the OP's question exactly; a proper empirical study on various dimensions of attributes.
Everybody here should read this.
Mastery in software engineering needs to be defined very specifically.
An example, I used to build Flash microsites, campaigns and games for an ad agency. I had a very good grasp of the tech, built projects for large clients etc. If you looked at me at that point in time you could say I had mastery, but now Flash is gone, and I didn’t switch over to React and other web based tech soon enough so my “skill” is lesser than someone who purely started in that domain.
I don't see them engaged in "heroic" coding marathons to desperately get something fixed or delivered; they already fixed it weeks or months ago.
Knowing that the correct answer to almost every question is "it depends".
Knowing how to somewhat-less-inefficiently produce larger-scale software that adequately meets the need. (Note that I did not say "efficiently", and I did not say "bug-free".) This takes at least some skill in dealing with people, not just computers.
A master will be able to understand the relationship between systems so he can reduce combinatorial possibilities and reduce complexity to a minimum.
In other words, a master will not shoot herself in the foot. She will also not let others shoot themselves in the foot.
Then, depending on what you do, you will need technical ability: -Understanding basic physics or economics or whatever you interface with your software. -Understanding of low level systems (assembler and C) -Understanding of different paradigms(objects, functional) -Understanding of very High level systems (Lisp, Haskell) -Understanding of serialization(parsing and binary) -Understanding communications. -Understanding data handling. -Understanding interfaces(like command line, desktop or Web). -Understanding standards.
But this depends on the particular work you need or want to do. You will be stronger or weaker in different areas.
But complexity handling is common to any area in engineering.
Competence is a better term. What it looks like is being able to make things happen quickly with minimal errors or problems.
Unless you’re talking about the most extreme cases ..... Fabrice Bellard and John Carmack .... genius developers who are true masters. There are few of these.
Here's a incomplete list of topics that I is worthwhile:
Creating a data schema (sql, no-sql, api based, whatever). This seems simple and obvious, but it becomes very hard very fast, especially when you need to model a complex subject that you know nothing about. Consider being hired by a hospital to model their records, or an insurance company. You need to talk to many people and understand how everything they say and do relates to everything else.
Project management: You may not want to be a project manager, but you should familiar with scrum, agile, waterfall, lean, and kanban. At least you can have an informed opinion when you argue for or against how things are done.
QA: You should know how to write a test plan for a large feature or application. This includes unit tests, manual testing, deciding what areas need more effort because getting it wrong is very bad and what is less important. This may include security and performance testing.
Architecture and design: How would you design a new application? How would you add a new feature to an existing application? How do you communicate this design to a team or many teams?
How to evaluate new things like languages, tools, libraries, technologies, etc. This isn't just reading a couple blogs, this is doing small scale test, proof of concepts, looking at the code to an extent, getting opinions for others you trust.
Finally, communicating. Software engineering is a team sport. You need to know how to write design documents, test plans, maybe architecture plans, progress reports, etc. If you can't effectively communicate your designs, plans, or even why someone else's thing is bad or good, you won't be able to convince others that you are right. And remember, you need to not just convince your peers, but also non-technical people, such as managers, VPs, and many other.
Schemas apply to more than databases. That may be less important for drivers, where the developer is also the domain expert for what they are implementing, but not necessarily so for avionics software. Lets saw you are implementing a black box. Are you the developer also a flight expert that knows everything that needs to be recorded and how each data point relates to each other? Not just in the sense of the hardware, but everything an FAA investigator wants to know. Probably not. So you need to work with pilots, FAA, and other non-developers to understand that domain.
Knowing how to construct a data model by talking to the experts of that domain is not just for databases.
Through-out my career, which is now over four decades if you include the software I wrote as a teenager (before I got paid to do it), I've continuously found a better, more elegant (which can almost always be read simpler or clearer) way to tell the computer what to do.
So my non-snarky view-point is that there's no such thing as mastery but rather that it's the proverbial journey. There is however a baseline at the other end ... if it's not functional and maintainable, there's a definite lack of mastery.
As an art teacher, I have seen many young artists go on to have great careers shortly after leaving school... but most of them burn out or fail in one way or another. Conversely... their aged teachers have acquired character, depth and stability but have in the process lost their wildfire.
- Should know many tools to the level where they know their strengths and weaknesses. This allows them to select a useful collection of them for any new piece of work.
- Should produce incremental project changes which all either improve the value or reduce the cost.
- Should balance all the improvements to produce useful, working software with a reasonable cost. For example, producing super-secure software with no useful features, featureful software with a bajillion bugs, or bug-free software with a gigantic cost are all bad.
Though in software engineering we have a lot of prior examples of successful and unsuccessful projects, we also have lots of open frontiers. And in some ways, it's much harder to "know" that an engineering approach or paradigm is "right" than it is to know that a theorem is true.
How can we make types track the "important" invariances of a system? How can we convince ourselves that a distributed system can guarantee certain properties, or that a modification to that system doesn't break those guarantees? If I build a homomorphic encryption system as a service, how would I build debugging tools for it? Acknowledging the halting problem and its cousins, when can static analysis tools make useful, meaningful predictions about programs?
In the bronze age, you could build an impressive stone tower. Sometimes you could even make it stay standing. But I don't think there were any master civil engineers.
Therefore software engineering, is solving problems with logic, within a set of constraints.
Where constraints can be; business requirements, hardware limitations, development time limitations, performance limits, solution simplicity, release frequency... Etc
Mastery of software engineering is efficiently solving problems, and optimising for the constraints that matter...
You wouldn't ask what it looks like to master the practice of medicine today because there are dozens or hundreds of specialities and roles all of which contribute to medicine.
Firmware engineering and 'agile' website development are very different practices with different assumptions, requirements, etc. I wouldn't expect someone to master both web development and video encoder development just like I wouldn't expect a doctor to be a master pancreatic surgeon and develop vaccines.
On the other hand there is a 'journeyman' level of software development and engineering skill that carries over between different development areas. It's definitely not algorithmic interviews though.
This is like saying you can't have a favorite type of rectangle because some rectangles are squares. If you're a master of neurosurgery, a reasonable person would also consider that to be a master of medicine. Just as I would expect a master neurosurgeon to not be a master dermatologist but be able to master dermatology much faster than I could as a non-medical person, I would similarly expect a master C++ developer to master Javascript frontend development much faster than a junior dev, let alone a non-software person.
Figuring out what the irreducible complexities of a problem are and at what tradeoffs these complexities could be addressed in different components of an application (including handing them off to the user).
A few corollaries of this are:
- Masters will write less code, as they will eliminate most of the complexity when there is time for this.
- Masters can more easily put the complexity where they want it, having more control over trade-offs between performance, usability, flexibility, etc.
- Masters will have an easier time estimating the required work needed to implement a project, as experience gives them a good intuition for what the irreducible complexity of a problem is.
- Masters will rarely tell you a "right" solution except for simple problems. Their answer will depend as much on your specific requirements as the problem you're trying to solve[1].
- Masters will often fix problems by simplifying code. Refactoring will be a natural part to their coding workflow, not a separate "housekeeping" activity.
- Those who hide complexity behind more complexity are not masters. Those who always fix problems by adding more code are not masters.
[1] It could be argued that "add to numbers and do it fast" is a single problem and that's not wrong. Requirements like "speed" and "usability" are really just another aspect of the desired outcome of a program, but it's rarely thought of as part of the problem to be solved, thus I am considering these constrains as separate from the "core" of the problem.
1) Knowledge of the core theories of computers: core ideas of computer architecture, good understanding of computational complexity (how long a program will take to run), networking...
2) Knowledge of software technologies: Python, Javascript, Java, C, Docker, Functional Programming, Object-Oriented Programming...
3) Knowledge of software engineering concepts: Software architecture, software process (agile, requirements gathering, testing), software complexity (and how to minimize it through good architecture and processes), general engineering principles (redundancy, fault-tolerance, solving problems early in the dev process...).
To me at least, the one that is neglected the most is the third one, "software-engineering concepts." Being able to think about systems at a high-level and well at the detail-level is one the most useful skills for an engineer.
If you can break your problem up into smaller, independent pieces, you can pretty easily manage complexity throughout.
By having a good domain model (which to me means context-agnostic and represented approximately in 3NF/BCNF), you automatically clear up most of the nasty interfacing between components. Also, having a domain model in a normalized form enables much easier querying for any required dimension of fact (i.e. SQL/LINQ/et. al.). The methods will almost automatically follow from a good domain model.
Being able to develop good models of the domain and manage incremental complexity are the hallmarks of a master software engineer. You should note that neither of these fundamental skills requires the use of a computer.
Ward Christensen did it once as well, to the annoyance of my late friend Lloyd Smith, may he rest in peace.
I will try to avoid doing that myself, specially I will avoid implying I am one, but I think it's possible to come up with a few criteria to help you "judge" the level of mastery one has in software engineering:
* can create a project from scratch without pondering forever about each and every choice, while picking reasonable choices that, together, approach optimality given the circumstances (notice that this depends mostly on the team/company involved so there's not one right universal answer).
* can break up the work in manageable pieces so that progress can be made (regardless of how many people actually work on the project) and measured easily. This is really hard and can only be done properly by a practitioner (meaning, no, your PM can't do that for you).
* can establish a good base design that facilitates achievement of the project's objectives (e.g. if performance-focused, then architecture might differ significantly from a more maintainability-focused one).
* knows to apply the right amount of testing/automation (too much or too little can be damaging).
* can focus on the task at hand, avoiding unnecessary work (many great technical developers fail to do this due to their preference for harder challenges that may not be the right ones to address).
* can communicate well (because knowing it all but not being able to pass the knowledge on or convince others will cause most of your knowledge to become inaccessible).
Notice that to be able to effectively rate a person or team against each criterion is itself something subjective sometimes, so you won't get a clear-cut answer no matter how you look at it... but I am sure that there are ways to use metrics for each one, though we all know the problem with metrics (people will game it), so yeah, it's difficult to figure this out. Just make sure you don't fall into the trap of judging people solely on subjective grounds though as our natural biases are quite crippling (see psychological studies about human bias) and will cause you to misjudge others, sometimes disastrously.
2. Think in the abstract. Implementation details are just that, details.
3. Ideas and concepts can be reused, code reuse (in business logic areas, not technical support) is rare.
4. "Design is about constraints" - Charles Eames
5. Understand the foundations, finite state machines, what REST as a style actually means, why Nouns are more important than Verbs, don't reinvent things...
6. Learning never ends.
- High performance (in terms of efficiency in using computational resources)
- Reliable i.e does exactly what we want and doesn't crash. This also includes clear documentation.
I think to reach there, some of the things one may want do are;
a) Learn TLA+
b) Study hardware (processors, memory)
c) Study software, protocols like tcp/ip
d) Build experiments & ship
So, maybe you'll find that to be a helpful practice as well. It's worth a try.
You introduced a new tool to your workflow. The consequences of that was
- Lower productivity initially - You where forced to think about how you create/edit your code
So, what made it work for you?
- Thinking about how you structure and edit your code. This gave you some clarity around how to think and solve a given problem. Longer you practiced this, better you became at it.
- Learning a new tool made you a better at certain things (Say how you manipulate large amount of text?). Better you became, more confidence you derived out of it. More confidence resulted in taking up more harder problems, and the cycle continues.
- You have started thinking about your tools. You spent time and energy in learning it. This has broadened how you think about software development. This in turn has widened your ability to look at the software problems.
I am not sure what I wrote resonates with you or not. But this is what I took away from you response "Making that change has improved my thinking"
I feel, it's not the choice of the specific tool that made a difference, but the result of changing your regular habits that VIM bought about in your way of writing code. I am sure, a person who changed to emacs or Jetbrains also would have seen something change :-)
What I think it did for me is two things:
- the splits allows me to have multiple files (aka buffers) open at ones time and this might decrease the number of times I have costly context switches.
- the vi way of manipulating text is faster and more efficient in terms of steps needed to complete an edit. I believe this reduces the cognitive load of mundane tasks and lets my mind stay focused on the actual problem I'm solving.
VI may not help everyone obviously. But I think it opened a flood gate for me.
Also the initial lower productivity wasn't really that bad so I don't think it had too much of an impact.
But I'd say a master is simply someone you have a lot to learn from.
I'd say you have mastered the discipline when even though you know the limits of your abilities and you are eager to learn, it becomes difficult to find someone to teach you something new.
The ability to judge risk in terms of time and money.
The ability to judge the amount of security that is needed (a productive level that is still safe).
To think long-term, in terms of avoiding complexity buildup over time, and in terms of avoiding vendor lock-in.
To help others get up to speed.
To stay productive over time.
To stay open to learn something new.
knowing what to Google.
So the master to me is the one who knows which tools for which job !
I'm just a devops guy but my current project is like that, where our lead architect has really impressed me by drawing up the design of a very scalable system. My task is merely to implement and manage kubernetes for it and we have other devs who are using java, .net and golang to write the components.
This impresses me a lot how our lead can come up with such a design and make it work from paper to practice.
It is SO common in this industry for people to literally not know what some software is meant to do, and just start typing code anyway. There's an entire software philosophy, "agile", based around that. If it IS known in advance what the software is meant to do, the quality of software that can be created is just astounding.
You can be a masterful manager, can deliver much more value than a single programmer, etc. But the engineering of software occurs primarily in the thinking about and directly working on it in its specifics.
It's not like he turned 50 and suddenly stopped coding. I'm talking about a real nerd, a person who codes even on their free time.
And he still writes code in other projects, he's a consultant, but in our project he's not touching any code, only advising. And he's 50+ now so he has 15 more years until retirement.
Of course I've seen people here work beyond retirement because our field is so easy on the body.
A person might have zero experience except Node.js yet still show high mastery of functional programming, mixed OOP, design patterns, compilers, etc.
Setting any threshold on languages just sets up a meaningless metric.
If you have these skills there is a very high likelihood that you can do most other things that are thrown at you
Most of the other answers have all these high level wishy washy specifications, I thought I’d take a more brass tacks approach.
I suppose this list could be called necessary but not sufficient. The question is what does a master “look like”. If you can only work with one or two languages you are by no means a master.
Wisdom is to acknowledge uncertainty and that estimates are uncertain in nature because the information available at the time is always incomplete.
- Symfony framework
To answer this question, one must first operantly define what mastery is. So what is mastery? To start, mastery is something on which one can be on the path towards, yet mastery is also a thing which has been achieved. So we need to talk about flow along a path and achievement of the objective.
PATH
The path to mastery is characterized by humility and child-like exploration combined with an intrinsic drive to integrate learning a and take paths that make individual sense to one on the apprentice’s path towards mastery. It is concerned with thoughtful action and adaptation. The ultimate focus is on the seeking of truth (an intrinsic reward) rather than and sometimes at the detriment to receiving external rewards (accolades, financial renumeration, and other materialistic artifacts). Such sacrifices and focus is made early in the mastery process and pay off once one inevitably achieves it. This is a pattern common in most individuals we would consider to be masters and the difficulty lies in staying on such a path despite internalized intuitions and the listening to an internal voice/drive pulling us away from society, the majority of who are not on the path at any given time. Mastery is high risk because it a deviant process often focusing on long term rewards rather than instant gratification or even short-term survival.
ACHIEVEMENT OF
What can a master do? Why is that beneficial?
Mastery is the seamless ability to fuse the intuitive with the rational while performing and understanding at a high level of competency in a given discipline or domain. It manifests as as such a deep internalization of a domain that one is able to spend cognitive effort on seamless cross-pollination between other disciplines, sculpting domains at will. Mastery is only achieved through embracing the humane, which means internalizing the fact that what makes us human is the blending of the rational (a step-by-step-step process) with the intuitive (a distributed and probabilistic process).
MASTERY IN SOFTWARE ENGINEERING
What is engineering? What is software? What commonalities do they share?
Given a definition of mastery, we see that the question of what engineering and software are next to be defined. These three questions should serve to form a foundation for discussion.
Engineering is the process of designing reliable processes and systems given constraints on resources, tools, and available thought patterns. It is also fundamentally concerned with risk management on the path to achieving an objective.
Software is a system which interfaces with either humans, machines, or both to achieve an objective. It serves as a medium for sending conceptual communicative signals across, within, and between various levels of abstraction to include physical computing done at the level of hardware.
As an aside, it must be stated that the delineation we place between software and hardware is an arbitrary though sometimes useful fiction to allow us to get tasks done. With this being stated, such a simplification is often over-used to our collective detriment.
The master software engineer understands hardware and software deeply at both the intuitiveness rational levels in order to design systems and process to specifications and with clarity to both machines and humans alike. This ability to provide and communicate with clarity is especially important when we consider that our colloquial models of machines themselves are over-simplified, a fact that one will quickly become aware of when trying to design applications across platforms that perform “low level” file system operations. In this case, it becomes apparent that even operations such as opening and closing files, tasks which one would think are stable and solved problems, have such variability that cross-platform development is more challenging than it should seem.
Not only does a master software engineer have the ability to communicate at different levels of abstraction of hardware and software—such a master also recognizes that communication is the operant word. Communication is signal propagation, filtering, and more, occurring at the personal, interpersonal, and organizational level as well. The master software engineer is seamlessly able to apply all principles of software development to psychological phenomena as well. To such an individual, there is no difference conceptually from a root pattern perspective.
That's big commits, you're not supposed to do that. Masters commit individual characters that replace entire modules flawlessly.
and then stops.
He understands that the universe
is forever out of control,
and that trying to dominate events
goes against the current of the Tao.
Those are creations of exceptional masters.
For smaller projects take a look at nginx, Tokyo cabinet (a DB), redis, and, perhaps, python's requests and Clojure (one man's effort to unify obvious things).
Again, these are masterpieces, not just merely good things.
There are a variety of positions one could take once reaching enlightenment: 1. Domination. One could choose to be controlling of the entire field and use it for selfish benefit. 2. Dismissive. Once you reach the pinnacle it becomes apparent that both humanity and the society it has created are horrid and aren't worth interacting with. 3. Appreciation. One could choose to be thankful for being able to reach such a position and to respond by generously using the mastery to use it for the good of humanity. 4. Constructive Legacy. One could choose to form ones own group around that mastery to preserve it beyond themselves.
I believe the dismissive position is the most reasonable and hence most likely to be chosen by those who reach true mastery.
All four positions regularly occur and you can see it readily in masters of respective fields.
The way a master appears depends on their reaction to having gained mastery.
I think the dismissive position is also most likely because I believe there are far more masters than we ever notice. They are not noticed because the majority of them don't bother to use their mastery in an obvious way.
Why should a master bother to correct those beneath them?
If a master interacts with society they have an uphill battle to being treated with the respect they deserve. For the true master there is great value in going their own way and rejecting society.