My guiding principles after 20 years of programming (2020)

I call this, Insight by Progress. I.e. the further you progress into a problem the more insight you gain.

Personally I have never 100% solved a problem before writing any code. For me it really the other way around. By writing the code I better understand the problem.

I guess what we're dealing with here is that different brains solve similar/same problems in different ways. I guess its also the reason that advice such as "Never start coding ... unless ..." doesn't work for everyone.

I agree. Coding a solution is my main approach to understanding a problem better.

While it is possible to completely understand the problem before writing a solution, it takes much less time to just build a prototype, analyze it to see what mistakes you made, rinse and repeat.

Note that the third principle of the "unix philosophy" is[0]:

    Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.

[0] https://homepage.cs.uri.edu/~thenry/resources/unix_art/ch01s...

If you read ahead two sentences:

> You need to progressively go through the code-test-improve cycle and explore the problem space till you reach the end.

So the author seems to agree, somewhat contradicting his own point.

If like most of us you are working on legacy (as opposed to greenfield) then the existing code is a part of the problem space. Coding (and debugging) is a good way to understand that problem!

This one is weird but I agree with the gist of it. You shouldn't be coding until you understand the problem and have a high level design planned out in one form or another.

Definitely agree. I think some people find it controversial these days with agile etc but i think making prototypes is a severely underrated skill and practice. Not all problems are obviously tractable from the outset by pure analysis. Sometimes you need to build something out of cardboard and rubber bands (figuratively) to see if it will work. Was a topic I wrote an essay about some time back (https://www.machow.ski/posts/galls-law-and-prototype-driven-...).

This is why a good IDE is so important. Since I use Jetbrains products (Rider, PhpStorm) I don't worry about refactoring anymore. This results in a very agile way of working.

I believe people underestimate the power of a good IDE.

Completely agree! This is my number 1 point on my own "I've developed SW for decades, here's what I have learnt"-post:

1. Start small, then extend. Whether creating a new system, or adding a feature to an existing system, I always start by making a very simple version with almost none of the required functionality. Then I extend the solution step by step, until it does what it is supposed to. I have never been able to plan everything out in detail from the beginning. Instead, I learn as I go along, and this newly discovered information gets used in the solution.

I like this quote from John Gall: “A complex system that works is invariably found to have evolved from a simple system that worked.”

From: https://henrikwarne.com/2015/04/16/lessons-learned-in-softwa...

I think that line may be more focused on eliminating any known ambiguities in the requirements rather than trying to determine any unknown ambiguities or tackling edge cases and bugs.

> While I agree with this in general, I find that to reall fully understand a problem, I need to attempt to code, or at least formulate, a solution to it.

Me too. Sometimes when I don't feel like I really know how to solve a problem, I'll just write some really hacky code to try to get an answer. Just anything that moves the problem forward.

Then once I understand what I want to do better, I either throw out that code and start clean. Or I just refactor the heck out of it until it is in a good state.

Yeah, it could just be me, but I prefer to make two false starts, toss them, and then get it right on the third attempt rather than attempting to whiteboard the problem for two weeks.

Not only is it more interesting to me to try three different ways to tackle a problem, but I have been burned when the two weeks of whiteboarding missed something and I'm back to having to iterate anyway.

To be sure, I do a little whiteboarding, but generally it might be about 2 hours or so of sketching out ideas, major structures, code flow ideas.

I generally was nodding along to most of the author's points though.

I definitely have grown to divorce myself from my ego and always try to not only try to shine the spotlight on my younger coworkers (new engineers) but try to give them "ownership" of key pieces to allow them not only some sense of autonomy/ownership but a sense of pride as well.

That does go slightly against some of the author's points about collaboratively working on a project. Engineers need a part of the code (let's say an image cache manager, as an example) that they can "own" though in order to grow. You don't want an engineer to always have "training wheels" on. (And frankly, I think this is one of the things I dislike about code reviews, I think it dis-incentivises autonomy.)

The team though, let me be clear, is the most important part of any project, not the individual luminaries. The "team" though, to a degree, needs to have engineers who feel an ownership stake in pieces of the product.

(FWIW, I have probably 35 years of programming experience, ha ha.)

me too...

I usually so a vertical proof of concept where I tackle each step I understand in a unit-test. ofc it's no unit test- the test framework just serves as an quick entry point where I can test different approaches next to each other or different versions/variants of each approach.

like a repl with the benefit of an easy view of my "history" with the ability to step back in time.

That's not coding the solution though, that's exploring the problem.

Obviously it's helpful to formalize the problem! This is no different from manipulating mathematical equations on a piece of paper. =)

This is pretty similar to the "pantsers vs plotters" debate in creative writing. Do you need an outline before writing a novel? How much of one?

I think this is common for many programmers. Programming isn't just a way to implement the solution to a problem, it is also a way to find solutions to problems (by working them out in code). Often times, programming can also help you figure out what the problem is. Programming isn't just about implementing things, it can also help with thinking.

the 'beautiful' thing about this, is that it's then impossible to convince product people and "stake holders" to authorize enough time and resources to improve the solution afterwards.

In my experience they'd only relucantly agree if and only if it's "do this or die". Which then happens under time-pressure or under stress factors of other kinds.

"if it already works, why fix it?" (because it's making developer's lives much more difficult than they could be? --"but how does this increase revenue?" --errhmm... because incresaingly difficult to maintain? but they don't care about this "that's your job")

This idea is the source of Fred Brooks' observation: "Plan to throw one away. You will anyway."

In order to implement a good solution you need to understand the problem well. Often, the only way to reach that level of understanding is by trying and failing to implement it.

These days, with modern refactoring and incremental development, it's less throwing away a whole program and more gradually refactoring it until little of the first code remains but I think the observation still holds in many cases.

I think this depends on the scale of the problem. For small, algorithmic, problems, the kind you would see in a coding interview I find it far more efficient to solve on paper, or on your head, before getting bogged down in code. The difficulty in these types of problems is not understanding the issue but coming up with a solution. For much larger problems, just getting you head around the problem is very challenging and you need some scaffolding, in the form of half-backed, but still precisely described solutions, simply to provide ballast to your thought process.

Analysis through synthesis

In a way, coding can be thought of as two distinct things.

One: coding is a way to play around with ideas, equivalent to a back of the napkin calculation, a diagram in the sand or a pencil sketch of a part. We use code in this scenario more as a way to write down our thoughts and process. But rather than writing in just pure English and writing an essay or a set of Todos, we write more detailed "specs". "Do this three times or until this value is zero" is also written in code and perfectly understandable in code too.

Two: code is used to run by a computer to produce an outcome. In this case, the back of the napkin sketch IS the part but more thought out.

Society is so used to there being two material objects between the "plan" and the"final product" that we forget that code is different. With code, the plan and the final product are of the same form. Of the same physical space.

>> Never start coding (making a solution) unless you fully understand the problem.

> While I agree with this in general, I find that to reall fully understand a problem, I need to attempt to code, or at least formulate, a solution to it.

I'd go as far as to say that programming is a good medium for expressing poorly understood and sloppily formulated ideas. Oh, wait, someone else already said that...

I'd add developers as a liability, notably bus factor; if you have only one dev that knows tech X or product Y, you need to get rid of tech X / product Y, or hire and train more people that know tech X / product Y.

Don't keep adding new technologies to your tech stack, it increases complexity and bus factor. I'm saying this as a warning as well because I've witnessed numerous projects where they switched to a new technology or tried to rebuild their tech stack, only to go back once the consultants and self-employed folk got bored. They hired external people to accelerate without having them focus on handover and building up their own IT department.

> E.g.: maybe don't output results reliant on a HashSet order - sort them.

I would much prefer shuffling them in that case. Less expensive and reduces the size of the public API. (Returning items in a specific order, whichever it is, still is a sort of promise.)

> "all observable behaviors of your system will be depended on by somebody"

Ugh, a former team I was on had that to the extreme. We were customizing open source software for internal use. A lot of operators had (over years) developed scripts that would just SSH in and use a CLI to interact with the software, using regex to parse output. We once thought we were safe adding a new line to the output rather than modifying an existing output line. Nope, someone somewhere used a multi-line regex.

Kicking everyone's automation out of the shells of those boxes was a multi-year project that is, as far as I know, still ongoing.

There are publicly accessible MIT lectures about A&D and computing performance. Part of the taught rationale is that performance behaves like a currency. Some thoughts about this:

The more you have of it, the more you can afford to do. This is interesting to think about. There might be stuff we don't do, that we rule out because we don't even think we can afford doing them. A slightly milder effect here is that we tend to make things more structurally (architecturally) complex and expensive, just because we don't even think of making them fast in the first place. Both of these effects impose real-world limitations and costs.

Secondly, if you don't have enough of it, then you tend to be constantly distracted and limited. Anyone who went through financial hardship for a time knows about the mental, physical and social tax this imposes. I think it is useful to think of performance that way. Or to turn it around: if everything we do and compute was incredibly fast and reliable (also an aspect of performance) then how would that change our behavior, well-being, productivity?

Side note:

Garbage collection is often used as an example in the way I describe above. One can "afford" to use it, or not. But I think this touches on a rather special aspect of performance, which is a broad term to begin with. It affects the overall memory footprint and variance (GC pauses) but doesn't necessarily affect overall latency and throughput, which are more generally applicably aspects.

"Performance serves usability" is, to me, another very strong reason to do at least some prototyping very early.

Valuing performance has indirect effects on other things: you're forced towards better models, better SQL queries and better validation/verification of data.

My opinion is that too often we say "good enough" performance, let's optimize later if this degrades overtime... And there we are, sometimes creating time-bombs of performance degradation just for the sake of delivering features the faster the better... When you have to clean up and refactor other people mess because it's a clusterf*ck of "just works", you begin to appreciate some basics rules and "dont's" about doing things in software

> But putting performance last contradicts the high priority of Usability in the list, because Usability strongly depends on performance.

Well that's why usability is its own item. Whatever is needed for usability is ranked as important by the list. If indeed your users find that their button clicks are just so very fast you need handrolled assembly to keep up, then writing that assembly is vitally important. Otherwise? Probably not so much. Performance for its own sake is last priority.

Now obviously there's a coherent argument that in general performance isn't considered enough when evaluating usability. That's totally fair, and probably true.

Gather a bunch of co-workers and lobby for 15 % time! You should absolutely have some time for pet projects at work.

I don't know about every single day, but I've worked professionally as a software developer since 1998; and I've been writing code for fun outside of work since I got my first computer around 1985.

Work is when someone tells me what to do or where/how to do it, which is why they usually have to pay me obscene amounts to make it happen.

Once you've settled into a programming job there is actually very little that is new and intellectually challenging. If you don't do anything else you'll become obsolete and atrophied.

To have pet projects is to break out of this daily slow death and to keep learning and being intellectually challenged while keeping up with the industry.

I still (after over 30 years as pro) develop stuff for fun on the site. I don't see that as work (and I don't see my work as work either these days either).

This one's very personal, it's only a good thing to do if you genuinely really enjoy it.

I used to be a maths teacher and I love maths, and would do it for fun outside of my job. That in turn often helped with the job itself, and certainly helped to keep my interest in the subject alive even when 90% of the job was teaching fairly basic stuff that wasn't particularly exciting.

I'm now a software engineer - I'm much happier in my job, but I don't love writing code enough to ever want to be doing it outside the 40 hours a week I'm paid for. I'm slightly jealous of the people who do, because I know first-hand that it's a great way to do better at work and keep feeling fresh and interested in what you do, but forcing yourself to do it doesn't bring any of the same benefits.

Readme based development sounds like a great idea!! Thanks.

Internally where I work we use RFCs if we want to propose a new feature/addition, it helps bring others in to skim the doc to find out if there is anything they need to add. It's not too bad, I quite like this

With big projects spanning across years, the "problem" is often a moving target. Adding new features alters the problem space dynamically. The best you can do as a maintainer is to be mindful of the original problem that the solution was architected for, and extend the thing without crossing certain boundaries and assumptions.

This point makes literally no sense to me. He says don't start coding before you fully understand the problem, and then he says the way to explore the problem space is coding (the code-test-improve cycle).

By the way I agree with the last part. Sometimes the best way to understand the problem is writing a half-broken solution for it. You just have to be aware that what you're writing probably won't solve the problem, and make sure all the other stakeholders are aware too (that's usually the hardest part).

I think it's key to think about the wording here

> understand the problem

> code-test-improve cycle

I think you're first trying to understand a non-programmatic problem, a business problem, a user problem. Then you start to explore the solution to that problem. Yes, you'll learn more about the "problem" as you solve it, but the cycle explores how you're solving the problem.

It should be easy to talk a pointy-haired boss (PHB, see Dilbert) into "readme-based development" (RBD). In fact, you can probably get funding for training, aka pizzas.

That's because RBD is basically Amazon's "write the press-release first" methodology and no PHB will say "we won't use something that Amazon says makes them successful."

The pizzas come from another Amazon saying.

I think you both oversell it. Debt, as the metaphor, implies borrowed something. You could be borrowing effort from pulled in libraries. You could be borrowing future maintenance time. Both can be valid choices. Both can be success or failure.

So, if you have the time now to not borrow, don't. But if you can leverage growth to have more future resources to pay back on debt, you should probably do so.

How much cocaine is reasonable to do then? And how frequently?

That ordering just states that you should not sacrifice security for simplicity. As long as two solutions are equally secure, the simpler one is better, but not when simpler one is less secure.

> Simpler solutions tend to be easier to rationalize/debug, leading to systems that are more maintainable, more reliable, and generally more secure (see OpenBSD).

Yeah, IF they are more secure, THEN use simpler solutions.

Totally! Simplicity is core to the rest of the others in the list.

The other weird one is finance near the bottom. Typical engineering bubble style thinking... make sure your solution actually provides value and that people will pay for it! Otherwise you're wasting your time solving the wrong problem! The importance of that cant be understated!(unless your goal isn't to provide value of course)

I guess it depends on what "decisions" we are talking about. I took it to mean design decisions, in which case Security and Simplicity are generally orthogonal.

Plain text passwords are a lot simpler and less secure than salted, hashed passwords. Using bcrypt to hash your passwords is a lot simpler and more secure than some home-grown, self-implemented hashing algorithm.

Also as it says he is talking about priorities, if something is simpler and less secure then choose security. If something is simpler and more secure then hey double win.

I keep telling people this: show me the data structures and I can make a pretty good guess about the algorithms that are used. Show me the code, and I still won't have much of an idea exactly what the data should look like.

In modern OOP, with code and data all mixed up and algorithms being injected at runtime with vastly different effects, things can get unreadable very quickly.

Well you can extend the tooling to suit your needs. As Steven Covey says, sharpen the saw.

Here’s a fun little story that’s stuck with me for years because it demonstrates this in a very hacker way.

(Forgive the lack of specificities, I tell this as a parable)

A fancy new office tower was opening downtown. Around this time, office space was at a premium so they soon sold out all their floorspace, and the project was considered quite successful except for one small problem; the architect seems to have installed slow elevators.

Tenants began to complain that the ride up to their high, and very expensive, offices was taking too long. These elevators were stupidly slow.

The building management frantically called around the big elevator companies, getting motors upgraded as fast as possible, but the complaints kept coming and in the end no one could quote anything less than $100mil to structurally alter the building to house bigger motors and more shafts.

Then one day an independent construction contractor showed up who offered to do a retrofit to fix the issue for only $10mil. The desperate building management decided to try it.

So the guy took $500k of the money to buy and install a bunch of mirrors in the elevator booths which had been sombre (and expensively) wood-panelled. Sure enough, the complaints stopped coming. (And the contractor got a $9.5mil payday)

The problem wasn’t that the elevators were slow. It’s that people got bored in them (in a world before smartphones). And what do people never get bored of doing? Looking at themselves in the mirror.

Of course, it’s annoying to save the equivalent of 95% of the cost but not see any of that money. You just had to deal with budget cuts that really should have killed any solution. Still a cool party trick I guess.

I’m from the same world as you and here’s how I understood this:

Any function that’s not pure should be a ~class~ component.

ie, when you have a bunch of functions that operate on the same kind of struct/set of state, and in fact you want to prevent other functions from messing with it (you want to make it internal), you group them in some way. The OO word for that is a class. React is a component; which are much more composable because we ditched the stricter concept of inheritance of OO classes.

Hell, the es6 class system is really just syntactic sugar around Brendan Eich’s nifty prototypal system. If it weren’t for the weak typing and lack of std.lib, JavaScript would be an amazing language.

…which is to say it would probably be Go haha

I disagree. For a huge majority of projects and teams, it is infeasible to build everything in-house. The way I see it, you can either

1. take an extremely impractical amount of time reinventing the wheel in house, or

2. take a (IMO a relatively much smaller) hit when you fail to take the 10 minutes to read the breaking changes on the dependency. Read this, make fixes, and everything is working again. If you hit a major issue like this with your own tooling, you're at the mercy of your team's ability to quickly build a fix.

A refined version of this point may be something like "only use dependencies which are very well tested, documented, list known issues, and have an active community of development and releases" - don't just install any package or library because "it appears to solve my problem"

yep agreed.

I've often thought about what interesting findings you would get back if time and people were infinite and all those little things you thought, 'huh? thats weird' and put it in a queue somewhere and a few days later it had been thoroughly investigated and findings documented by some smart person. I'd expect you'd find some weird networking problems for why ssh hangs sometimes, misconfigurations, waste, security vulns etc. So many things we skip over day to day bc it would be a distraction.

Why can't someone be a well-rounded individual with hobbies and a life outside of programming, and also have time for pet projects?

Is every job a social service?

It takes 20 years to learn to compromise? I would expect these kinds of lists to change very little after the first 10-15 years.

I realized at about 10 years into my career I would rewrite code as it was easier than learning someone else’s implementation.

Now that I’m cognizant of this I first thoroughly review other implementations to try and understand why decisions were made, ultimately saving me from repeating the same mistakes.

Inconsistent documentation is also a problem.

While living in Italy i learned this idiom "Chi va piano va sano e va lontano". It means "who goes slow and steady wins". It has fundamental impact on my work career. Now I don't rush to learn new tech stack but take things more slowly and immerse myself in it.

"The only way to go far is to go right".

I can see where he's coming from. I started around the same time he did and from my point of view from my experience, perhaps I can further explain how I see it.

>Performance is last ? Code length is more important than performance? That's wrong. Also correctness is not mentioned at all surely that has to be in that list.

Depends on the scenario. Early in my career I was maintaining a scheduling DSS. You would enter in some appointments and it would consider a bunch of rules the users would create and find an appointment with matched resources. It was very cool and I really got into it. One of our test cases was scheduling 12 related appointments all at once. After some tuning, our system could do it in about 10 seconds, give or take. I decided to give a prototype a whirl with different ways to store and process everything (in memory) and got it to schedule these 12 related appointments in under a second. I showed my boss and he was impressed, but ultimately said, "the user's don't care about the difference between 10 seconds and 1 second, it's 1000x faster than doing it on paper." When the OP says performance last, I feel this is what he means. Some developers tend to tunnel vision on picking the best set of algorithm and caching mechanisms above all else, allowing the product to face delays when no-one cares about the difference between 10 seconds and 1 second. Of course, this is dependent on the solution. If you are google or Netflix trying to serve millions of users, performance matters. Most of the time, great is the enemy of good in this regard, especially in enterprise apps. Also, you can always tune trouble spots later down the road.

>I bet this person has had a problem with a dep a few years ago that caused a lot of emotional damage and the response is to never use 3rd party deps where possible.

I write stuff in house whenever possible. I've been burned by external libs plenty of times, and there is one I have in my current codebase I wish I didn't. It depends on the quality of your developers, for really good teams, this isn't a big deal at all. I use third party stuff for more specialized libs like image conversion and whatnot. The benefits of in house libs is they only do what you need, so they are tight and when something goes sideways, a smaller codebase trying to satisfy 1 use case is easier to understand than a large lib trying to satisfy 10 difference use cases. Plus, you don't always have access to the source code of third party libs.

>> Good code doesn’t need documentation >This is impractical at the end of the day you'll need to have something for the user to lookup for reference.

Yes, I'm torn on this. I've written plenty of documentation that nobody has ever read but me though. If he means "code with good comments," I'm ok with that. External documentation outside of good tight specs is hit or miss. A lot of it is just check boxing.

>Most devs don't get to decide this they will be told to do X by someone in a large org and given a brief description of why. Refusing to do your work until you fully understand a problem will lead to dismissal.

I don't disagree this happens, but if it does, you're probably in an environment that doesn't respect you, your work or your growth in the company. Get out ASAP. I always explain to my devs why I'm doing something; mainly so they won't write some funky code because they don't understand the problem.

> Good code doesn't need documentation.

Here's the full quote from the article: "Good code doesn’t need documentation, great code is well documented so that anyone who hasn’t been part of the evolution, trial & error process and requirements that led to the current status can be productive with it. An undocumented feature is a non-existing feature. A non-existing feature shouldn’t have code." Maybe you're just objecting to the first part, but he isn't saying "don't document".

Yes, if you apply for a SWE position.

experience != skill

Experience is merely the prerequisite for skill, it doesn't cause it.

If course, and algo whiteboard sessions. It's even harder for seasoned devs since they graduated ages ago.

It’s my litmus test for idiots who think they have something useful to say.