Three Tribes of Programming (2017)
josephg.com
josephg.com
In Computer Science, there are 3 kinds of problems:
1. The problems that Math gives you: algorithms, data structures. These are the problems you think you're going to solve and sometimes you do, but often we just need a few really smart programmers to solve this and the rest of us can just use those solutions.
2. The problems that Physics gives you: hardware, device drivers, network latency. Sometimes you do systems programming and work close to the hardware. But once again, once one engineer has solved this, most of the time we can just use those solutions.
3. The problems that other engineers give you.
And that is where you're likely to spend most of your time: gluing APIs together, reading code and documentation, and arguing about code formatting.
This is why in so many ways computer science is a social science. Certainly "software engineering" is. Everything you do is going to be about communication. When you write code, you're not just communicating with the compiler and the machine, you're communicating with other engineers. You're going have to craft smart documents, emails and collaborate. Any nontrivial solution will require these skills, so you better start investing in them.
Much of programming is encoding what you want to computer to achieve and is therefore a rather long process, but not necessarily hard. Precisely specifying how a GUI should look has a substantial amount of necessary complexity, and writing that out doesn't fall into 1, 2 or 3.
What on earth do you mean by this? It sounds rather like advocating just doing the minimum you can, for the sake of harmony.
What I speak is about individuals trying to compete against its own team to defeat them in some unnecessary priority rather than collaborate and play all together.
For example, I’ve seen diffing / recomputation code operating in O(n^3) or O(n^2) where O(n) or O(1) is possible and necessary for reasonable performance. I’ve also seen extremely poor use of network requests (multiple serial requests) that happen because the developer doesn’t understand network latency and networking performance (in this case with a single data center in the world).
I can't tell you how many times I've seen people "make it work" and create a mess that has to be rewritten later with much greater difficulty than if even the slightest amount of forethought had been given before hand.
The focus for me is really building appropriate abstractions over the work I do so that people can focus on improving only the vertical slice they need to.
https://blog.codinghorror.com/regular-expressions-now-you-ha...
I am a maker. I build things for people to use and maintain.
This puts me at endless odds with the hackers. All three groups suffer from Novelty Seeking and Risk Taking (aka Adrenaline Junkies), but the hackers seem to get it the worst.
My hatred of tools that fall apart the third time you use them goes back to childhood. I don't want to put my name on any of that shit.
And yet...
I'm also quite comfortable and capable performing emergency field repairs on objects real or virtual. I think the difference is I know this is temporary. Single use. When we get back to where there are tools and supplies, we should take it apart and fix it for real. The hacker declares victory and moves on to the next problem.
Where I disagree with the breakdown here is that the makers are also poets. It's just that the medium is different, so it's hard to compare. I think it's safe to assume that the same is true of the hackers.
I remember once coming across a 3rd party web API whose documentation stated you could configure something like 50+ settings with it. Only about half of them actually changed the configuration, the other half I had to actively go in and start doing things like shutting down the service, updating various files, and then restarting.
When I realized this I just thought to myself, how in the world does your pride as a developer even ALLOW you to ship something that bad.
Another example is PoSH (Powershell SSH). The author just randomly changed how he does things, and changed the defaults, so one day things that worked before just stopped. You now had to configure it to have the old behavior. WHY!?!
Then I realized that if you issue a shutdown command over SSH via PoSH it would hang until the timeout was hit. I then found myself maintaining a custom version of PoSH to work around the issue. I even went as far as to describe the issue, and my fix, for the author, but to this day I don't think they ever actually fixed it. The day when we can natively use SSH from powershell can't come fast enough.
The point being that things are always so damned brittle. And the worst part is that I don't think it gets better the lower down the stack you get. I've ran into plenty of these types of issues in C and C++, it's a mindset and it drives me nuts.
If you have no empathy then you can watch them and dismiss it as user stupidity. Which is never an attractive look.
> I build things for people to use and maintain.
Maintenance. Maintenance. Maintenance. That's the key. It is the lack of that mindset which is killing our home world. Ultimately, it's just expression of not having to pay for your externalities.
It also kills software dev. If software devs were actually held to account for their legacy (software, hah), then we'd have a lot less shitty (and shitty+, shitty++, etc) software.
This is just a thought (and may be unpopular): practices differ wildly from one company to another, and between enterprise software vs hacky side-projecty node apps, nonetheless, maybe it's time to standardize software good practices across the entire industry and enforce them, making them not just "good" practice, but mandatory practice.
It's like saying that tents shouldn't exist and people shouldn't use bicycles even though they are great tools for certain jobs.
Nobody wants to pay for space shuttle levels of software engineering. It costs a lot in labor time to produce less results.
Edit: Maybe I should go around pretending hackers and poets have the moral high ground.
When you're young it's easy to derail your complaints by implying you are not smart enough to appreciate the magnificence of the code. I tend to take Feynman's position that if you're really so great, your solutions will be obvious.
I'm just saying that what companies sell as products should not be made in the "art" mindset. Is that clearer?
So you're dev ops?
Jokes aside, do you really have a passion for maintaining projects? This is something I deeply struggle with and any advice on the subject I would be deeply grateful for.
I thought I could deal with it and enjoy a project in the long run if it had no bugs, so I worked on some of the code the internet runs on, where even putting in one bug would make parts of the internet go down world wide. (Which I did do btw. Sorry.) I both hated and loved that job, but in the end, even in a "bug free" code base, so maintaining it was only feature requests for the most part, I still lost passion over time.
I switched to Data Science, writing code I know will be scrapped. I write mockups to analyze data. It's a lot of fun! But to me it's just running away from the issue.
How did you gain passion for maintaining code bases? If you have a story, it would mean a lot to me. I'd love to hear it.
As you point out, though, some code actually doesn't require maintenance. But at the same time, that's not always the code you think.
I don't. I have a passion for clearing roadblocks and that means different things at different times. Sometimes that means automation.
I think you parsed that sentence in a different way than I intended. I think a legitimate handoff is part of building a tool for someone. We have people who run the gamut from rent-seeking to dropping the mic and walking away.
Rich Hickey and Bret Victor here are squashed into overly simple (and/or wrong) categories whose descriptions go like "the best programs [...] formally prove correctness" and "beautiful code is more important than beautiful UI". Both have been very vocal against these points.
Jon Blow is definitely not (just) in category 2. "But one of the reasons his last game (The Witness) took so long to write was that he wrote his own engine instead of using something off the shelf [...]. I understand". He'll probably have a heart attack reading this deep, deep assumption that creating his own thing slowed down his unique game rather than enhanced it.
As someone who does OCaml (and who maintains ReasonReact), shoving it into the category of "poetry is more important than execution speed" and "code over UI" is exactly what we're going _against_. It sucks to emphasize pragmatism, compile & runtime and UI, only to be shoved back into the "poetry" category.
I get a weird feeling criticizing a blog post that mostly gets the right idea across but then goes e.g. "don't worry, we get that you're a poet and care more about that than real-world execution and we empathize with your different perspective" or "we get that you don't prioritize writing simple beautiful code". Like, I _do_ care and these folks _don't_ abide by what you're describing. There's gotta be a name for this.
But extremes are useful to define a scale. In your work on ReasonReact, how much time do you spend thinking about the ergonomics of the users? It sounds like a fair bit of your time and mental cycles go into that. So it sounds like you sit somewhere between the extremes of those camps. I don't think I would have been able to say that sentence without defining overly steriotyped, extreme positions.
And there's lots of interesting thoughts that follow from that - like, what happens when those motivations come into conflict? Eg, users care about feature X, which is going to be really ugly to implement, but desired. What do you do?
My personal answer to your question for me is "It depends. Lemme check what feature we're talking about here". My mentality isn't "nice, an occasion to exercise my principles", and judging from the talks two of the three aforementioned people gave, I'm positive they adopt a similar attitude: happen to take a stance, but don't actively seek it.
When I go over some codebases, I often find flavors of code that smell like someone tried too hard taking a stance. E.g. code that overly future-proofs, code that definitely needs less ad-hoc-ness, code that prematurity optimizes in an obviously naive way, code that tries too hard to use a functional/OO concept, etc. Note that these all belong to different categories in your post, but one thing they share is that they all feel like p̶e̶o̶p̶l̶e̶ ̶w̶i̶t̶h̶ ̶d̶i̶f̶f̶e̶r̶e̶n̶t̶ ̶p̶r̶i̶n̶c̶i̶p̶l̶e̶s̶ ̶s̶l̶a̶p̶p̶i̶n̶g̶ ̶m̶e̶ ̶i̶n̶ ̶t̶h̶e̶ ̶f̶a̶c̶e̶ ̶w̶h̶i̶l̶e̶ ̶l̶e̶c̶t̶u̶r̶i̶n̶g̶ ̶m̶e̶ ̶a̶b̶o̶u̶t̶ ̶h̶o̶w̶ ̶t̶o̶ ̶c̶o̶d̶e̶ folks shoehorned principles back into the project rather than the other way around.
So an alternative way of categorizing things is whether you go from the principle to the project, or vice-versa (aka whether you go from abstract to concrete or vice-versa). A result of that is that Jon Blow and Rich Hickey end up in the same category while Bret Victor and Alan Kay ends up in another (without speaking too much for themselves, and thanks to the nice confluence that is HN, I’d say that this categorization has been empirically verified if you check these people’s discussions with each other).
Also, as computing domains grow, you'll probably need to create many more categories if we cross-cut the way you did in your blog post; on the other hand, this alternative way of cross-cutting, I believe, would stay constant. But hey, that doesn't mean your way of cutting it is wrong. The alternative one might be simpler but more abstract.
Maybe this explains why many people in this thread debate on whether the cross-cutting is fair. If you ask the alternative question "which direction do you work from/to", you might get some better partitioning.
- The first tribe is mostly coming from Maths background, their favorite performance measure is the big O notation.
- The second tribe is coming from Physics/Engineering background, their favorite performance measure is expressed in SI units, most often a duration.
- The third tribe is coming from everywhere else, and their favorite performance measure is customer satisfaction, or in other words dollars.
This a programmer that models systems (real or imagined) into working computer applications. I suggest that in many cases, the other tribes largely exist to support the creation of these systems.
This includes the systems that track and support your money, taxes, utilities, services, scheduling, reservations and insurances amongst others. The systems that enable you to purchase and consume things and have them delivered within the context of a whole supply chain.
It also includes the modeling of control systems, from the software that runs an elevator, to an airport baggage handling system, to an automated warehouse.
Typically in these, the challenge is in abstracting the problem domain under consideration - the determination of what to produce, rather than how to do so.
Examples: Kristen Nygaard, Peter Coad
Favorite Languages: Java, C++, C#, Smalltalk
Hangouts: Currently suspended due to the focus on problems in the computer and data sciences, and computing infrastructure.
To use an example from the article, I think Jonathan Blow would absolutely argue that he wrote his own game engine because it was best for his user. Whether you agree with him or not, he is whole-heartedly convinced by his assertion that it could not be done in Unity trivially. Having listened to him a fair deal, and his feelings on games (vs. programming of games), I very much believe that he cares a tremendous amount about the experience of the game above quite possibly all else. So, while you can certainly argue that the way he chooses to implement that goal is misguided (I don't believe this, but an argument exists), I don't think its fair to give "The UI is more important than anything else" to the third camp and not his. I think he agrees with this statement, and feels that what gets you the best UI in a game happens to be low level work (to avoid frustrating lag, etc.).
Similarly, I think Bret Victor might take issue with this as well. Much of his work is centered on creating innovative UIs to help people think, and often against the (current) abstract ways of representing knowledge. In particular, from the maker perspective, he cares deeply about empowering people to make things -- less so I would say than some sort of mathematical purity.
All this to say, I think there may be less difference in goals than we think, and more difference in what we believe influences those shared goals.
So I don't think it's a stretch to say a performant alternative to C++ with a game engine out of the box could have an impact which has broader reach than speedier development of his next game.
* Source Code: Your code is clean enough for you. Your top priority is thorough documentation and intuitive API design.
* Execution: Critical around bottle necks, like large batch processing and build times, otherwise it doesn't matter. Iterate and optimize based on feedback from your devs.
* Correctness: The program should function exactly how it's described to function in documentation. If something unexpected happens, the error is clearly and concisely exposed to the developer so that they can understand what they did wrong.
* UI: Usually not a thing, but when it is, your users are developers and they should be able to figure it out ...
Personally, I'm a gamedev, super in Camp 3. I've worked with a lot of folks who could care less about product and polish, but love making their colleagues lives easier. It's pretty similar to Camp 3 in the sense of "making for your users", but the skillset and priorities are very different.
Favorite languages: TypeScript
Hangouts: npm, github, anywhere open source code is distributed
I'd add to add: Fav languages: Whatever is already installed and easy to deploy and maintain/debug on your target system. For linux systems, this might mean Python, Bash, Go.
Additional Hangouts: Forums about AWS/GCP/Azure tooling, Hashicorp projects, Config Management Software projects (ie, ansible)
There is still an interface that the developers are using, whether it be command-line based, graphical, an API, or something else entirely. And hopefully it is well-designed!
You will also find that many of the people who are interested in Clojure are also interested in Haskell and vice-versa.
In fact, they share enough that it causes a tension because it is a little "too close for comfort". i.e. the concepts they share are quite strong, and the concepts they differ are stark by virtue of being polar opposites.
No, they aren't they same along every axis, but they share enough axes that the crossover is quite palatable.
I've come to view many supposed divisions between people this way. There aren't homeless and successful people, so much as there are people that are homeless right now, or people making lots of money right now. People can certainly swing between both. Nothing is forever.
I find this perspective helps me to appreciate more my present situation in life, as well that of others.
Getting back on track, we should find value in the diverse perspectives in programming and find things to learn from them. One of my pet peeves is the programmer who does not have a learning mindset. That toxic behavior transcends the divisions laid out in the article, and is to everyone's detriment.
Extreme mathematical programming is turning theorems into code; Metamath is an example of this, as is Coq. Nobody uses this code because it isn't meant to be used outside of a mathematical context.
http://us.metamath.org/index.html
Extreme hardware hackers are either designing missile guidance systems or mirrors which turn your face into a plot of its Fourier transform. If their code explodes, it's either doing so in accordance with a design document taller than you are or it isn't a problem because exploding mirrors are even more fun than the normal kind.
Extreme maker programmers, as the site says:
> Taken to the extreme, this world view doesn't value the beauty in the engineering itself.
Taken to the extreme, this group actively derides good design because good design takes time and Real Artists Ship. Real Artists Make Deadlines, too, and Real Deadlines Are Set By Marketing. You end up with janky crap that looks good but leaks your password to anyone who breathes on it, as sold by people who think being proactive with lawsuits is a good substitute for keeping your data secure. Security Through Suing Anyone Who Says It Isn't Secure, in other words.
I fancy myself more of a maker (I like making useful software) who happens to dislike like Python and Javascript. And also Java. Because the lot of them are inelegant and I hate reading and writing them.
- Tribe 1: the art component wins.
- Tribe 2: the science component wins.
- Tribe 3: the business component wins.
Your life is easier when you belong to tribe 3. I also believe that you can't choose your tribe.
I believe that the three groups mentioned in the article map pretty nicely onto this framework and that, while you may not be able to choose your (maximum) stage, you can over time progress into higher stages.
Version 1 is what you write fast, to ship it quickly. Corners are cut. It isn't even held together by duct tape and baling wire. It is glued together with bird spit. But it gets the crucial first dollar of revenue. Version 1 is programmed by hackers.
Version 2 is what you write more slowly, painstakingly, to replace version 1 before its innards inevitably spill out onto the pavement. Version 2 is paid for by version 1, but is programmed by the math-poets, the software "engineers". It is very elegant and clean, but doesn't always do what the customer needs. There is zero technical debt. People weep when reading the code, it is so beautiful. But it still doesn't export the management report to Excel yet.
Version 3 is what you get after version 2 has been in production so long that many of the conscious design decisions introduced by forward-thinking architects have been patched out and replaced with simpler things that just work, and are easy to maintain. The ultimate goal of version 3 is to replace itself with a shell script, and then take a nap until a new requirement or feature request comes in. It does everything the business wants, but everything is a massive wad of cruft, and there are legacy bits that no one wants to touch.
The number reported by the software has nothing to do with this. A lot of software never gets out of version 1, or goes straight from 1 to 3, or skips 1 and starts at 2.
OK, here's a problem. Read a file of text (the name is to be passed in as a command-line parameter). For each line, if it begins with a number, print out the number.
I'd use Perl, and it would take me one or, at most, two minutes. It would take me seven lines, only because I put my curly braces on their own lines.
Now, I picked a problem that is a better fit for Perl than for Haskell. But that's kind of the point - there are problems that are a better fit for other languages than for Haskell, and when you hit one, you're better off writing the program in a language that fits better.
mapM_ putStrLn $ filter startsWithNum $ lines $ readFile (args !! 0)
where startsWithNum = ...
Still, I agree with your point -- no language can be optimal for every problem. perl -ne 'if (/^(\d+)/) { print $1, "\n" }'
Put the list of input files at the end of the line. grep -oP "^\d+" grep -Eo '^\d+' fileThis one contrived example does favor Perl, and will beat Haskell by a small margin. But as soon as the user says "Oh, sorry, I was mistaken, I needed the number and the next letter" or something like that, Haskell is on top again. As Haskell is one of the greatest languages for handling "Oh, sorry, actually..." requisites, and Perl one of the worst.
As long as the "actually..." only needs a small change to a simple regex (as in your example), perl probably wins the "actually..." game as well until you pull in enough libraries that the Haskell looks a lot like the perl. It's when the regex starts getting overcomplicated and the Haskell breaks out parser combinators that Haskell probably moves into the lead.
As you say, though, the margin perl maintains is small.
I just thought the express claim being made significantly oversold it. Whichever approach you take with the Haskell in the example under discussion, modifying the Haskell is not easier than adding a handful of characters to a simple regex.
Once I've finished building something, as a "maker", I want to be involved in communication with customers and to see how my work is being used. If I've done the hard work of programming something that's useful to others, the reward I need is to see how useful they are finding the thing I've made.
However, for a professional developer the reward for finishing a programming task is just "more programming". I can see how this would appeal to "poets" and "hackers" - it gives them the opportunity to "write more poetry" or "do more hacking" - but not to "makers".
I am a mathematician who makes things and uses math to understand how the code executes. Execution is not an implementation detail to me.
Category theory and type theory don't fit cleanly with assembly language (the implementation detail).
Additionally at the very fundamentals of mathematical CS: the Church–Turing thesis doesn't describe anything about the real world. It says that the implementation of a true lambda machine vs. a turing machine are essentially the same thing, the thesis does not describe how it's basically impossible or extremely hard to build a true lambda machine.
I'm just saying that a big portion of the math doesn't describe the real world implementation details.
As I said the Church side of the theory doesn't have a real world machine equivalent. We have machines that move things and save things but we don't have a physical machine that represents the concept of a function call. What we instead have is a Turing machine that emulates a function call, not a true simulation of it.
I think it’s weird as fuck that you are telling me what I am.
Up and Down the Ladder of Abstraction is still one of my favorite examples of an interactive webpage, and is beautifully elegant in how it integrates with and supports the text.
http://worrydream.com/LadderOfAbstraction/
It's cleanly designed as does exactly what it needs to. Hard to beat, IMO.
I notice I do strongly identify with the Poet/Mathematician, I'm just not one in practice. I do love elegant code, algorithms, data structures, code that's concise, logical and easy to read. I love that many programming languages have become more functional. I wish I could do a project in Scala. I wish I understood Lisp.
In the end, though, I don't care as much as the real Math/Poet. I don't care for Scala's over-the-top Turing-complete type system. I don't care for proof of correctness. I care that my code works and looks good and is easy to maintain, and joyful to work with, and those are practical matters for makers.
I also appreciate the work of hackers, but mostly because their compilers and VMs and OSs make it so I don't have to worry about those things. I've programmed in C, but I love not having to worry about memory allocation anymore. I want my algorithms to be efficient, but I don't pretend I can reinvent an existing algorithm and implement it better than the standard library has already done for me.
So I don't see the strife, I see different roles. Makers make the end-user applications, but to do so, they use languages and programming principles designed or influenced by the Math/Poets, and they run on efficient and secure platforms built by hackers.
We need each other.
Hack, I started as a FE intern in Amazon, originated Amazon Kinesis, built tools for managing google's data center network software, Borg core and user library, and now other stuff.
My work as a programmer constantly change. Am I in any tribe then?...
I would say you're not a poet though.
In modern app development our computers are fast enough that this kind of thinking isn't really important any more.
Thinking it doesn't matter is how we end up with things like instant messaging clients using billions of bytes of memory and billions of CPU cycles to receive a dozen bytes of text and display it on the screen, and when every application on the system wants to consume like that, none of them can, and the result is unhappy users.
That said, from my experience it often turns out that the simplest code is also the most efficient and elegant as well as being likely to be correct, so from that perspective I'm essentially valuing all 3 tribes.
Rust has been fun because it allows me to indulge in the good parts of #2 without totally losing my sanity. It's not great for #3 though.
I don't think most people fall into just one camp, but these three dimensions make a really useful "personality test" for programmers, and do a good job explaining conflicts like those mentioned by the author.
I mean, "cool". You have ideas, but everybody has one, I have mine and I guess there will be someone with even multiple ideas on the matter. So we will not ever be able to agree, point.
So why bother?
I know what you're talking about though... Lots of blog posts talk about their own ideas about the industry that are largely irrelevant. But if you touched all the technologies he has listed in his write up you'll see he's completely correct. He's not speculating about something. This is reality.
Why?
I think the author probably thinks Ada's type system is more powerful than it actually is (or is just thinking of SPARK). Ada is basically just verbose C with a slightly nicer type system and better aliasing rules.
I'm seeing a new generation of kids coming out of school who are just pissing on those techniques: only functional is mathematical, and the rest is outdated, intractable garbage that causes software crises and meltdowns.
Though I'm not really sure I'd call maintaining invariants and pre/postconditions mathematical elegance per se, it feels like good engineering. Like you've built something that's solid.
I guess I'm not a person, then? All three are great languages. (Well, language families, in the case of Lisp.)
nice terminology and nice categories
There may need to be a fourth tribe? I write my programs to accomplish tasks for the user, but I use tribe A and tribe B languages to get there as they make the computer 'smarter' while I'm designing and coding.
My programs should encapsulate and execute whatever work needs to be done as quickly, safely, and correctly as possible so the user can get on with their day. After all, programs exist to accomplish tasks for people, whether directly (my app code) or indirectly (memory allocators).
I try to design/code for the scenario of a new user who knows logically what task they want to accomplish, but not how to get the computer to do that task. My programs should guide the user like a travel map. You are here, you want to go there, and you need to do the following things to do so.
This means if they are missing or have malformed input, I tell them what input they are missing and what the program expects it to look like, along with trimming leading/trailing spaces.
Nobody likes a picky program, like the accounting system running on that one AS/400 in the basement that MUST have a CSV with exactly 15 columns and no more than 32 characters per column (and don't even think about using commas anywhere) or the whole thing silently fails after 12 hours.
As the user learns how the program works, they don't see/need the guides any longer as they know more and more of what they are doing. They want to speed up. Binding keyboard shortcuts to every menu option enables advanced users to breeze through the program from muscle memory.
So what languages do the mythical fourth-tribers use, those who are fine with standard libraries and not sorting binary trees, but don't countenance Electron web bloat or ever-vanishing discover-ability/accessibility/readability sacrificed on the altar of 'mobile first design'? Over the next 3 years, I plan to learn the following high, medium, and low-level languages so I can be productive in each domain, that way wherever my users are, I can empower them to rule their machines like a boss.
F# is great because of features like pattern matching, static types, and immutability, which all combine to uphold the 'pit of success'. Other languages tend to uphold 'check it all at runtime, cause I won't check it for you'.
Rust is a low-level language without the apparently rampant memory leaking remote code execution undefined behaviors of C or C++. Writing secure C or C++ code is just about impossible for experts, let alone curious beginners like myself, so why even bother? Rust it is.
Due to its trendiness (okay, more like more active community) Golang just took over Free Pascal for my next up middle level language. It sits between native low-level Rust, and JIT high-level F#, and it features easy parallelization and has just one or two ways of doing things, reducing code complexity.
Free Pascal and Erlang are on the list too for historical and always-stable-forever reasons, respectively.
The poet is a rarity.
I work in a golang shop and when I tell people that json and unions are hard to work with in GoLang because it's missing an essential part of the poem (the coproduct) I get blank stares. Not one person knows what I'm talking about.
I think of it as the next level. Everyone starts off as a maker or a hacker, then if or when you get really good you discover the poetry behind all of it. Most people don't ever get that good though. All they are concerned about is the next implementation detail
Makers are a dime a dozen. In the end everyone is "making" something so everyone is a "maker" in a sense. It's just the "makers" described in the article don't know how to hack or write a poem so they just concentrate on delivery time.