Mastering Programming (2016)
facebook.com
facebook.com
Being a fairly seasoned developer myself, I kind of intuit what they mean, but I could very well be projecting what I think I am doing right into each of these, and conclude that I'm a master programmer... yay!
The enlightened sit nodding along, while the unenlightened go away confused or misled. Positioned this way, half-understood ideas frequently do more harm than good.
TDD is a great example of this.
In this case, though, I think the article does a fairly good job - it highlights productive ways of thinking that will incite useful self-reflection ("Do I do this? Could I do it better?") rather than half-communicating ways of doing.
It's the same with maths. It's even worse when the concepts are explained in a circular way. Go to wikipedia and search for a random mathematical concept. I guarantee you that most of the terms used in the explanation have wikipedia articles that link back to the concept you looked up in the first place! Frustrating beyond belief. You need to get a foot in the door and then it's easy, but before that it's hell.
It's also that most people who teach just don't have Feynman levels of understanding. Mostly we are all kind of groping in the dark, and we all participate in some kind of cargo cult.
I also think there's a deep problem with how people want to learn math, and other abstract concepts. Look at how frustrated many students get by notions like imaginary numbers, groups, and monads. These things provoke some serious anxiety and confusion because students want to understand what they "really are" in some clear and straightforward way, when really all they are is hard-won abstract definitions that have proven interesting or somehow useful.
In a way it's disappointing that math books need to have "exercises." Why can't they just explain in a comprehensible way? I guess it's because the topics of study are abstract relations that only sometimes have some analogy in the "real world" (of course, some mathematicians believe that the structures of mathematics are in some sense the most real of all)...
I think this puts a lot of things into perspective for me. There will be a blog post about some topic on HN, and it seems that everyone in the comments just gets it. Yet I'm over here like, "Yes. I know some of those words. Maybe I'll get the concept a bit more next year." Time passes. I'm still none the wiser.
Functional programming was one of those weird topics for me when I first started programming. It's pretty straight forward on the surface. Some X has to go in. Some Y has to comes out. If it didn't originate inside the function, then it does not get touched. Yeah but why? Who cares if I have a variable outside of the function scope that gets modified? Didn't I declare it globally specifically to have access to it anywhere? What's the point of all these stupid rules when languages give me the ability to do so much more? But all these people can't be wrong, can they?
So I shoehorned it in. I tried to make everything I wrote as functional as possible. I started creating convoluted, over-engineered garbage that was needlessly complex. Nothing felt right about it.
Then I actually tried it. I sat down with a somewhat pure language to learn functional concepts, and it clicked. I see why people extol the virtues of functional programming. Like you've noted, not understanding the reasons behind using functional programming can lead to a convoluted mish-mash of cleverness that doesn't belong.
I'd like to see a blog post on just that point. well said.
Thank you! I'm glad I'm not the only one who thinks this. There's an abundance of articles explaining how to "best" test, usually using an overly simplified example (I'm looking at you, `add(a, b)`). When you try to apply it in the real world, things break down because no one explains how to deal with the icky practical issues.
Despite all these resources online teaching you how to test, I feel that how to test well in real-world scenarios remains tribal knowledge.
It's even worse (and quite common) when those who don't understand are the ones nodding along. Knowing when you really understand something, and when you don't, is such an important milestone for people to get to, and quite hard to teach.
The key to this is realizing that once you decide a problem is at a certain scale, the fix needs to be at that scale too. So you might have a data problem, a code problem, a workflow problem, a design problem, an architectural problem, a team problem, a project problem, an organizational problem, a leadership problem, or an existential problem. (on the level of the company)
Each type of problem has to be resolved by the resources on tap at that level. You fix data problems by editing the data. You fix code problems by changing code. You fix leadership problems by changing leaders. You fix existential problems by changing jobs.
Becoming a more effective programmer involves understanding at a deep level how everything interlocks and moves together. If you can't solve it at one level, escalate to the next level and try to solve it there.
So you might have a code problem that you get pushback on fixing, this makes it a design problem. So you get the stakeholders in a room to come to a decision about what the code should be doing. If they can't come to a decision then it's an organizational problem, so you go to the leader for a decision. If he can't make a decision then you've got clues to a hidden leadership problem at the company.
Problems should be instantly solvable once you get to the level they're really at. You don't escalate though until you know enough about it to make a short presentation to the next person in the chain.
Once you get above the workflow level, you're in the realm of politics. Political problems are defined by the fact that multiple people in the organization care about it and you need to get sign-off before you go and unilaterally change something.
I think in this case I agree with GP that it is possible to 'solve' a problem at one degree removed from the source if there is no way to actually solve the problem at the source.
So for instance, lots of people write code to scrub janky data from a partner (indeed, that may be part of the value you add to the system).
In order to fix a problem at certain Level, you have to go to 'Bottom Most' layer which is ROOT cause of the problem.
>So you might have
a data problem,
a code problem,
a workflow problem,
a design problem,
an architectural problem,
a team problem,
a project problem,
an organizational problem,
a leadership problem, or
an existential problem.
There's no way it's a data problem
It was a data problem
So the client complains about software behavior. You now have a client problem. You isolate the issue down to a problem in the data. You fix the data but the problem recurs. You now have two problems, a client problem and a code problem, that's generating the bad data. You go to change the code but find a design problem. You now have three problems. You have to solve the design problem so that you know how to fix the code, so that you can then go tell the client that his problem's fixed and here's a free month's service for your trouble.
Here is one thing I gained from this Analysis. At times we encounter 'a Code problem' partly caused by ' a workflow problem' and partly caused by 'a Design problem' .
We may do a CODE patch fix for time being (ex. production bug), but having it documented as '40% Workflow problem, 60% Design problem' will help to consolidate all these 'contributing percentages' to come up with permanent fixes at a Later time.
well sometimes problems have workarounds, otherwise the poor 100x programmer would have to solve all existential problems first. (maybe a 1000x programmer could do that, when he finally reveals himself)
Far more nuanced than Maslow's
A shitty team can produce great code - a great team can produce shitty code. A shitty leader can kill a great team. an existential problem can destroy the value of any data-set. etc...
overall I totally agree with you and OP, and thinking through this is a great idea.
These problems have to be tackled by management. Management has to change the scope and goals of the project so that the design team can better create deliverables for the engineering team to build.
Once you get to this level, generally the coders can only raise issues, they don't have the expertise necessary to actually recommend solutions.
About the only thing a coder can do if he sees something like this is to work really hard to actually understand what's going on, raise his concerns to management, and if he doesn't like their answer, get another job. Because chances are, if the project dies, so will his job.
for a novice, he might be enlightening
don't underestimate how much you know, or assume that everyone else knows it
As far as I can tell, she didn't avoid that trap, despite identifying it. Occasionally I wonder if I avoided it. There are so many ways for a project to fail and when you know too many you can become a nervous wreck. Over most of the course of the '10s I've noticed that I'm a lot more comfortable with certain classes of imperfection, because they are immaterial to the job at hand, or represent an unknown that we haven't tackled yet (if you don't know how something should actually work, making it 'perfect' might make it perfectly wrong, and then where are you?). It's like a chess game. Half the board is full of pawns to be sacrificed for some higher priority. You have a contingency plan for any threat, but you can't act on them all at once.
Probably everybody does this, but their set points are different than mine, so we argue over code reviews.
Classic middlebrow dismissal.
Even if the code is "messy" and given that JUnit is used by millions of programmers hourly without trouble, I suspect Kent prefers the latter to the former and which makes him a master programmer in my book.
I was picking out some new darts at a billiards store and me and my buddy were discussing what size, weight, etc. to get (i.e. "best practices").
The owner laughed and said the only thing that matters in picking a dart is how well you can hit the target with it.
I think these tasks are not meant for programmers in the first place. But indeed it's beneficial when programmers know how to handle those things properly.
I've always been of the mindset that if the code isn't a little "gross" it's probably not doing anything very interesting.
The less you start off with, the better your understanding will be, and then you should learn to expand your horizons with other programming paradigms: logical, functional, and many others.
Totally agreed on other paradigms not being teached enough though.
Please list a project of equal complexity that has been written beautifully from start to finish in your opinion. This myth of beautiful coding needs to stop! Code is not a masterpiece from Van Gogh, it needs to ship. This determines quality, not the usage of some obscure/mainstream/unreadable/oh-look-how-cool-I-am idiosyncracy of the language. If you ain't shipping, it's completely useless.
The code, imo, is some of the highest quality js I've seen. It is also incredibly comprehensive and easily as complex.
High quality != shipping fast. They are separate quantities, it is easy to have one without the other but very rare to have both.
My point wasn't to ship crap, it was about obsessing on quality over actually shipping (and cutting some corners). This is something I learned at my first job, we obsessed over quality and never shipped. Guess what happened? :)
We believe we are more like machines than like other people. But at the end of the day we're the same trumped-up monkeys smashing away with imperfect tools.
If you look at our fellow productive monkeys, they aren't using the prettiest tools, no. But the tools they are effective with have a certain beauty to them. They often have a single purpose, and they do it very well (the Unix model knew this 40 years ago). Nicks and dents don't matter, unless they're on the working surfaces, in which case they have to be repaired or replaced, so resilience in certain dimensions is an absolute must.
A master does learn to work with their tools instead of against them, but within reason. If a better tool is available, or they can modify the existing tool to fit them better, they won't hesitate for a second to do it. That is what is meant by 'A craftsman does not blame his tools.' If you know you're using a bad tool, that's your fault. Get better tools.
Details in this excellent summary comment by someone else: https://news.ycombinator.com/item?id=9848097
I specifically recommend studying the "Project Oberon" book which is now available as a PDF from Prof. Wirth's own site, I believe.
libfreetype is an example of a great library, clean and well documented...
beautiful coding isn't a myth, it actually works... write code like poetry and it pays off 10 fold... less errors, easier to reason about, malleable... the benefits are extreme... try to actually experience something before making unfounded claims... i've been writing beautiful for over a decade now... once you realize code is written once and read 100 times, you'll come to the realization of doing it clean and right the first time pays of in the future, which means less work and bullshit...
> write code like poetry
When a code base reaches a certain size it's hard to write "beautiful code". Please don't pretend that it is just as easy when you have multiple complexities/priorities. If you write so called beautiful code in complex projects, good for you.
I think that pretty clearly illustrates that Kent and Ward were wrong and Cedric was right. And yet I still listen to both of them about other things.
Nobody has it all figured out. You figure out what things you can trust a person about, and you learn from them, and you move on to someone else for some of the rest.
[edit] But I confess that I have a similar relationship to Josh Bloch that you have to Kent Beck. I can't listen to code advice by someone who wrote 2 collections APIs without understanding the basics of variance or apparently even the Liskov Substitution Principle. Throwing exceptions because you decided not to implement part of a public interface is an unforgivable offense that others have duplicated because if it was good enough for Josh it was good enough for them. He has done next to nothing to atone for that sin. You aren't an effective programmer. I don't want to hear you talk about the subject you so obviously know nothing about for 500 pages.
Having said that, I think he's talking about a crucial part of programming that beginners simply don't appreciate: how to break big problems into small problems. In other words, how to unstick challenges (and even find joy in programming) before writing a single line of code.
For example, make a list. Prioritize the items. Strike some. Put the simplest at the top. Figure out which item causes you the most anxiety. Chop away at the branches before attacking the trunk. Even intermediate developers often fail to see that the best programming takes place in the form of a plan.
Best to work on the challenging, unknown items first, especially if they form the core that everything will rely on. It gives you time to work out the kinks in the overall architecture. It's much harder to change those things later and harder to predict how long it will take. It's also easier to mock out and fake the branches.
I feel like I know what Kent is talking about behind most of these points, even though his words only hint at the deeper principle. He's trying to be too brief, but I can tell there's depth there.
For many of the points, I had to think a bit, and then it clicked, "Oh, he's talking about xyz. Yep, he's right. But his words don't really say that."
Kent Beck is not some AbstractFactoryStrategySingleton fool. I've learned a ton of insights from him, and I've seen him articulate insights that I had stumbled across and incorporated, but had not been able to articulate.
I'll grant you that some other talking heads often lumped in with Beck have been overhyped and their rhetoric has run ahead of their insight. And no one is perfect. But Beck usually has real insight behind what he is saying, and this IMO is no exception.
> Call your shot. Before you run code, predict out loud exactly what will happen.
M, okay, but what exactly do you mean by that? Or even poetic, kind of:
> Rhythm. Waiting until the right moment preserves energy and avoids clutter. Act with intensity when the time comes to act.
I have no doubt that each point has some very specific examples behind them and knowing them could actually be useful. But right now, you can only guess what was intended behind these #deep thoughts.
Maybe it was more of a business aspect/market timing for a new product?
You will sometimes need to work very fast in order to push your change while everything else does not change much. In these cases, it's important to be able to enter crunch mode, and it's equally important to alternate crunch modes with slower paced work to not burn out.
At least, that's how I see it...
In particular:
> Rhythm. Waiting until the right moment preserves energy and avoids clutter. Act with intensity when the time comes to act.
I've sometimes gotten bogged down in trying to pull off a major refactor or internal initiative that I had to abandon because it was at the wrong time. It's important to wait for the surrounding social context/business context/preliminary work to be in place before making a major change.
EDIT: to complete a thought.
> M, okay, but what exactly do you mean by that?
I frequently do a variant of this, although not always out loud. One way to debug a problem is to randomly vary various bits of the program until the problem no longer appears. Another way to debug a problem is to use the scientific method: develop hypotheses, then test those hypotheses with an experiment. I think he's saying to develop hypotheses, and then run code to test them.
This document clearly shows that "master" programmers focus on very different things than a beginner might expect. A beginner can get really excited and proud making something complex work without noticing the flaws. After years of debugging your own code you'll find out that getting complex things done is not really the issue, it is HOW it's being done that makes the difference.
Beginners tend to focus on new language features, premature optimisations, having unnecessary strong opinions about wether using a for loop or map or reduce etc.., using too many and buggy libraries, literally copy and use code from others without noticing it's flaws, getting pride from writing complex code that almost no one can understand, trying to beat the calculator, trying to beat Google with your own memory, over estimating the value of linters, CI, TDD, agile, what editor to use, etc.., etc...
All these things take up so much space and energy at the cost of what really matters if you want to do what a master does: write bug free code that is easy to maintain and a joy to read.
This.
Slicing: No. Cut the project into appropriate-sized slices. Cut big slices into sub-slices. Hierarchy is your friend.
One thing at a time: Yes.
Make it run, make it right, make it fast: Yes.
Easy changes: No good changes are hard once you own the result. If a change feels hard it's probably not good yet.
Concentration: That's one of two approaches. IMO either rearrange the code so you only need to change in one place, or rearrange your change so that the code only has to change trivially in any place.
Isolation: Depends.
Baseline: Yes.
Call your shot: Yes, but only because it means "know what you think will happen" which means "understand the codebase". If you're working on someone else's codebase, discard this and replace it with "try things and learn from them."
Multiple scales: Yes.
Symmetry: You mean 'factoring'. But yes.
Aesthetics: We define beauty as solving problems efficiency. If it's ugly but it's the best answer, we'll learn to think it's beautiful.
Rhythm: Yep. A time for all things.
Tradeoffs: Yes. You can't make a call til you understand the context.
Fun list: I guess that works.
Feed ideas: Never be afraid to ask what-if.
80/15/5 rule: Situational. If current low risk work is gravy spend 100% on that. If not, dig into R&D.
Stopped reading right there. My experiences have led me to believe that 9/10 Facebook posts of this nature are purely for publicity's sake and not being taken seriously by the author.
Anyone who says 'they get it' is likely just afraid they will be viewed as stupid if they say they're confused.
Anyone remember the story of the "emperor's new clothes?"
From years of watching master programmers, I have observed certain common patterns in their workflows. From years of coaching skilled journeyman programmers, I have observed the absence of those patterns. I have seen what a difference introducing the patterns can make.
Here are ways effective programmers get the most out of their precious 3e9 seconds on the planet.
The theme here is scaling your brain. The journeyman learns to solve bigger problems by solving more problems at once. The master learns to solve even bigger problems than that by solving fewer problems at once. Part of the wisdom is subdividing so that integrating the separate solutions will be a smaller problem than just solving them together.
Time
- Slicing. Take a big project, cut it into thin slices, and rearrange the slices to suit your context. I can always slice projects finer and I can always find new permutations of the slices that meet different needs.
- One thing at a time. We’re so focused on efficiency that we reduce the number of feedback cycles in an attempt to reduce overhead. This leads to difficult debugging situations whose expected cost is greater than the cycle overhead we avoided.
- Make it run, make it right, make it fast. (Example of One Thing at a Time, Slicing, and Easy Changes)
- Easy changes. When faced with a hard change, first make it easy (warning, this may be hard), then make the easy change. (e.g. slicing, one thing at a time, concentration, isolation). Example of slicing.
- Concentration. If you need to change several elements, first rearrange the code so the change only needs to happen in one element.
- Isolation. If you only need to change a part of an element, extract that part so the whole subelement changes.
- Baseline Measurement. Start projects by measuring the current state of the world. This goes against our engineering instincts to start fixing things, but when you measure the baseline you will actually know whether you are fixing things.
- Call your shot. Before you run code, predict out loud exactly what will happen.
Concrete hypotheses. When the program is misbehaving, articulate exactly what you think is wrong before making a change. If you have two or more hypotheses, find a differential diagnosis.
- Remove extraneous detail. When reporting a bug, find the shortest repro steps. When isolating a bug, find the shortest test case. When using a new API, start from the most basic example. “All that stuff can’t possibly matter,” is an expensive assumption when it’s wrong.
> E.g. see a bug on mobile, reproduce it with curl
- Multiple scales. Move between scales freely. Maybe this is a design problem, not a testing problem. Maybe it is a people problem, not a technology problem [cheating, this is always true].
- Symmetry. Things that are almost the same can be divided into parts that are identical and parts that are clearly different.
- Aesthetics. Beauty is a powerful gradient to climb. It is also a liberating gradient to flout (e.g. inlining a bunch of functions into one giant mess).
- Rhythm. Waiting until the right moment preserves energy and avoids clutter. Act with intensity when the time comes to act.
- Tradeoffs. All decisions are subject to tradeoffs. It’s more important to know what the decision depends on than it is to know which answer to pick today (or which answer you picked yesterday).
- Fun list. When tangential ideas come, note them and get back to work quickly. Revisit this list when you’ve reached a stopping spot.
- Feed Ideas. Ideas are like frightened little birds. If you scare them away they will stop coming around. When you have an idea, feed it a little. Invalidate it as quickly as you can, but from data not from a lack of self-esteem.
- 80/15/5. Spend 80% of your time on low-risk/reasonable-payoff work. Spend 15% of your time on related high-risk/high-payoff work. Spend 5% of your time on things that tickle you, regardless of payoff. Teach the next generation to do your 80% job. By the time someone is ready to take over, one of your 15% experiments (or, less frequently, one of your 5% experiments) will have paid off and will become your new 80%. Repeat.
The flow in this outline seems to be from reducing risks by managing time and increasing learning to mindfully taking risks by using your whole brain and quickly triaging ideas.
I am taking 'right' to mean 'correct', and not some lesser metric of quality, or an aesthetic preference.
For those cases where making it fast is the hard part, one could make the same argument, perhaps even more clearly. Having something that runs, but which uses an algorithm that will never be fast enough, could be the first step down a long dead-end.
"Right" doesn't necessarily mean "precise."