Let a thousand flowers bloom, then rip 999 of them out
gigamonkeys.com
gigamonkeys.com
Management has a vested interest in pushing the meme that there are no 10xers: it gives them an excuse for not paying these engineers what they're actually worth. The rest of us just go along with it because it assuages the inadequacy we naturally feel when we compare ourselves with someone like Linus Torvalds or John Carmack.
Clearly there are 10xers, and some of them even post here: Walter Bright, Bryan Cantrill, and Ted Unangst.
Edit:
I'm also not surprised Seibel would say something like this given his past demonstration of lacking social awareness by titling his book of programmer interviews "Coders at Work"--a title that one of his interviewees, L. Peter Deutsch, rightly objected to, much to Seibel's shock. Deutsch in 1984 co-authored the seminal paper on building efficient virtual machines for dynamic languages through just-in-time compilation and inline caching, techniques which ultimately gave rise to the JVM. He deserves an immense amount of respect for this and other accomplishments, and we do him and our profession a great disservice by publicly referring to him as a "coder" or anything else that elicits something less than awe from the general public.
Could you imagine doctors referring to some highly-regarded brain surgeon as a "cutter" or "butcher?" No, instead they talk them up as "brilliant" or even god-like, yet we do the equivalent with "coder" (and allow the media to do it too) all the time.
But the 10x gospel we see in software engineering is an entirely different beast: the claim is that there's a small group of people who "really get it" and that everyone else "just isn't made of the right stuff". It assumes both a kind of determinism (you do not become good, you either are good or you're not) and a hard dichotomy (there's no ordinary engineers, just bad ones and great ones).
Of course management would prefer to make ordinary engineers better than to let a couple of brilliant minds work their magic. But that's just common sense: https://en.wikisource.org/wiki/Why_I_Never_Hire_Brilliant_Me...
One thing I've noticed: the people who talk about 10x engineers almost uniformly believe they're either apart of that elite cadre, somewhat close to it, or will one day ascend to its ranks.
I believe that there are nodes in a network which have greater inherent value than other nodes, but can only achieve 10x value when working with other nodes of similar caliber in the context of the problem they are solving.
For example, Linus Torvalds wrote git in a month and in the process effectively dug the grave of svn, a project into which thousands of man-hours had already been sunk.
The "10xers don't exist" meme supports the notion that programmers are interchangeable cogs and justifies the ~$130k salary ceiling we run into without going into management.
Maintenance becomes ever more complicated and hard as time goes on because the complexity of the bugs goes up by orders of magnitude as does the complexity of the code.
I've spent more time adding a very minor bit of functionality to a system I wrote 5 years ago than I did writing the whole thing in the first place. Because it grows and adding new code or modifying existing code becomes more and more expensive.
How many man-hours have gone in to git now? And it's not as if git is that great anyway, we all know how bad the UX is for example and all the man hours that have gone in to masking that problem, it's just better than SVN.
Torvalds and Carmack is an interesting twosome to start with because from where I'm sitting, having never met either of them, Carmack is a dream engineer from every angle - and coworkers alike. If I worked with him and found out he made 10x my salary and got 10x the leeway I'd shrug, nod, and say well yeah, of course.
On the other hand, if I worked with Torvalds at a company that wasn't his encompassing baby, I think I would quit in a year as would many others. He can only get away with the shit he pulls because he is a singular genius. But most people who act like him are not, yet because some managers think they're 5x or 10x or whatever, they get indulged and coddled.
I've worked with enough assholes that I'll take a couple less productive human beings over a ninja-pirate-superstar prick.
There is nothing saying that a 100% increase in software productivity will cause a 100% increase in business value. Sometimes the relationship will be 1/1 sometimes 1/10 but sometimes 1/0.1 or even. 1/-x. It depends on so many things (how clear and adequate the goals are, the state of development, the context of the business).
Does anyone have a citation for this?
http://programmers.stackexchange.com/questions/179616/a-good...
Provisional title: "I used to be a 10x programmer like you, until I took an arrow to the knee"
(also, there are multiple different types of 10x programmer who are not necessarily cross-substitutable)
1. Engineers are not computers, so saving 5 minutes a day will not increase productivity by 1%. Most engineers work with passion, and don't look at the clock. They are focused on delivering a certain task, and they will make their best to deliver it, even with the interruptions, even if this will require them to work 10 minutes more at the end of the day.
2. The benefit of the tools is a clear win, but as these tools will not change so much over time, after the initial boost of productivity, the effect will no longer be noticeable. You can't increase performance each month compared to the last month just by using the right tools. So I think the EE team make sense to be created "on demand" and not as a permanent solution. Time should be a parameter in the effectiveness model
3. If you grow to a large scale enterprise, as the people are not computers and reaching agreement between large number of individuals is not easy, the bigger the number of people in the EE team the harder will be to approach solutions and test them.
I'd say giving your engineers 10 minutes more free time at the end of the day will give long-term returns too. Engineers are not computers, they need time to unwind after work like everyone else.
In other words, maybe you're both right and it's not about saving time per-se, but about cutting BS.
For the size and intensity of effort going on at Twitter, I respectfully disagree.
Once the team gets large enough, NMP ("Not My Problem" mindset) comes into play, and unless you have the culture from the beginning to always be making the development process better, and acquisitions and new management has never changed that, which is highly unlikely, then having a dedicated person or team to help make other developers jobs easier is a great idea- after the team gets to a certain size. What that size is depends on what is being done, though.
"My decision to take the plane does not negatively impact the environment, since the plane would fly with or without me in it."
Tooling cruft also increases the startup cost of some tasks. Where the amount of pain you will experience trying to do it may inhibit your desire to dig in.
The author was generalizing he wasn't assuming perfect accuracy. But in an engineering organization the size of Twitter the productivity gains of a dedicated tooling team add up fast. Much faster than you might think.
I think to spin it even worse, replace it with some sort of group crime. I joined the riot, because there would be a riot with or without me. I join the drive by shooting, because there would be a drive by shooting with or without me.
I think that would have less people agreeing with it than the plane argument.
after the initial boost of productivity, the effect will
no longer be noticeable
Right...
First, IME engineers aren't any more passionate about their jobs than anybody else. Even for people who are passionate, the majority of their day is unlikely to be 100% dedicated to the things they're passionate about. Nobody I've ever met is passionate about digging through their inbox in the morning, or going to scrum meetings, or doing sprint planning, etc.
Second, engineers are also the ones who get most upset and demotivated by the pointless waste of time. If you can speed up my compile by 5 minutes, then do it. If you can speed it up by 5 seconds, then do it. There's nothing the engineers I know hate more than wasting time on tasks that could be automated or made to run faster.
One of the distinctions I learned from Lean literature was that bureaucracies fall on a spectrum between controlling and supportive. I had honestly only ever experienced the former: rules and mandates to make me do what other people valued. But I came to realize that there are other possibilities. For example, when my team jointly makes a checklist we follow, that's a kind of supportive bureaucracy. I could imagine scaling that up, but it's certainly rare.
In this post is a hunger for power that makes me uncomfortable. Sure, any short-term mess can be solved by bossing people around. But I've seen places where centralized quality groups become strongly counterproductive. Every mandate they issue is intended to clean up a mess. But as developers become disempowered, they do less and less cleaning on their own.
Hopefully that won't happen at Twitter; there's a theme of wanting to build trust. But there's also the theme of ripping stuff out, of controlling the chaos, of whipping things into shape. That sounds much better to the person holding the whip than to those getting whipped with it.
GOOOOAAAAALLLL! Fail Whale.
GOOOOAAAAALLLL! Fail Whale.
http://highscalability.com/blog/2010/11/4/facebook-at-13-mil...
I like to think of this idea as group vs. individual velocity. Yes, you personally might be able to go faster by introducing new technologies - but everyone else is slowed down by having to learn it. Knowing which few of these technologies will be an overall win is the tricky bit.
Does anyone have anything that shows they don't exist? I'm starting to think it's something average engineers like to repeat to convince themselves that there isn't anyone really 'better' than them. Sort of a reincarnation of the condescending "there are no winners or losers" thing we do with childrens sports.
The top tier of developers are far more than 10x more valuable than the average developer. Not because they produce 10x more lines of code, or "crush" 10x as many bugs or sprint points, but because they build. better. systems. Period. They write better code. They build the right stuff. They promote better team cohesion. And they execute on difficult projects more effective.
And they don't just make one order of magnitude difference, often they make multiple orders of magnitude difference.
Here are three fairly common scenarios involving such "keystone" developers:
There's a large and crucial code base which is owned and maintained by an entire team, but there is one developer who has a much different relationship to the code than the other people on the team. They wrote most of the original code, though it has grown much since then, and they know far more of it than any other single person on the team. Tasks related to the code which would require much more time and often the collaboration of multiple team members can be done alone and with much less time by this person. If they leave the project will live on, and it may even flourish, but the development velocity and the capabilities of the team will be significantly diminished compared to when the key developer was around.
There's a core feature on a very important product that is in development, and there's one developer who seems to have an outsized impact on it. They've gone the extra mile to make sure the feature is designed and implemented well, they've put in the hard work to get the feature shipped with excellent quality (both in the code behind the feature and in the way it works and feels to end-users). They've made sure that the right work has gotten done and the most important core aspects have shipped. And the end result is a phenomenal product that attracts and retains users, engenders customer loyalty, and ultimately results in a massive amount of revenue for the company.
There's a tiny team or a startup working on an ambitious product. Despite their limited resources they produce something of significant value and quality, something that normally would require a much vaster number of average developers working for much longer to achieve a similar result. This leads to runaway success.
These scenarios happen all the time in the industry, and they speak to a gulf in capabilities between the most capable devs and the industry average. And it's not necessarily about "productivity" it's about getting the right work done, done well, and done efficiently. You put a top tier developer and a middle of the road developer on the same task for a week or two and the difference isn't that one has written a lot more code than the other, the difference is that one developer has something that works and something that you wouldn't mind keeping around.
This is another thing. Good developers learn and build from their past projects. Which means that over time the developer is getting better, and also what they're able to deliver based solely on their own work grows exponentially, because it builds and builds and builds, like compound interest. Whereas mediocre developers are building foundations of sand, they can only build so high before everything comes crashing back down and they have to tear down and replace what they've built before with something a little bit better. Mediocre developers tend to spend their time running in place.
P.S. Another huge factor is culture, mentoring, leadership by example, and work environment. Top tier developers raise everyone around them to a higher level through their example and their interactions with others. Learning from better developers is a hugely important way that devs get better at their job. And merely getting into a mindset that getting better is possible and valuable is important in its own right. Additionally, working with talented engineers, people we look up to, people we take inspiration from, etc. are important reasons why devs show up to work every day. When an "anchor" dev with a lot of talent leaves a team or a company it can trigger an avalanche of talent leaving the org as other talented engineers decide that the work environment isn't as worthwhile as it once was and they leave as well, accelerating the process until a lot of the talent has evaporated from the org. This is a very common process in the industry because the people who tend to be the most talented and the most intrinsically motivated tend to have the easiest time finding other jobs and tend to be the most sensitive to the quality of the work environment. Unless you're immersed in the work environment and sensitive to these things you may not pick up how drastic these changes can be until years down the road when the impact of a team that has lost all its talent finds itself vastly less able to execute on important projects effectively.
And while I've talked about a lot of "soft" subjects such as "art" and "work environment", these things absolutely have an impact on the bottom line.
I assure you, it is craft. It's much less in the realms of art, science, and invention, and much more in the realms of putting well-known patterns together to build something, sometimes even something new.
I assume you have never seen a modern woodworker at work?
Wood-working using the skills of working the lathe, the chisel, the rasp is fundamentally the same sort of thing as painting using brush strokes. And both skills can be used to create either a work of art or a work of craftsmanship. It's a matter of skill and intent. Fundamentally though, creating a painting, a poem, or a song is mostly craftsmanship, it's the application of skills to build something in a familiar way.
There's no magic breath of life that elevates a crafted work into the realm of art, rather it's the purpose it's made for and other factors which make that distinction. Something that is considered "not art" would typically be something that is completely unoriginal and has a purely practical purpose.
Art, on the other hand, tends to involve creativity and originality, it tends to not just be for a practical purpose but also to be evocative, emotional, and communicative.
You see, the method and skills of construction are not the differentiator here. Both a EULA and a sonnet can use the same medium (written language) yet while one is purely practical and ordinary the other communicates more than mere facts and details, involves some degree of originality and creativity, and engages the reader at a higher level.
Software dev is a very wide ranging field, so it is difficult to talk about it holistically. However, in my experience it does not fit the description of unoriginal rote work. Often there is creativity and originality, as well as subtlety and fine artisanship. It is a new kind of thing which does not neatly fall into either the manufacturing/engineering/"craft" or "art" buckets, and for that reason it requires different ways of thinking about the work and the workers.
The most important point here is that when you give craftsmen the task of making 20 chairs, they will do so. Some with greater efficiency than others. But this task is nonsensical when it comes to software, because it is only ever necessary to make one of a thing in software, copies are free (or as nearly so as makes no practical difference). Instead the equivalent task would be to make 20 different chair designs. And now it becomes more clear that we are not dealing with craftsmanship, we are dealing with design work. Creative work. The closest analog to which is art. And how do you judge an order of magnitude difference in design work? How do you objectively measure the difference between an Eames lounge and plastic patio furniture? Objectively you can't, but subjectively we are aware that the difference between middle of the road design and top tier design is much more than a single order of magnitude.
Let's put it this way: 10X engineers ship 10X better products. Some places still use the daft metrics you mentioned (LOC, bugs squashed, etc.), sure. I'd assume that a gig such as Twitter would know that a 10x engineer is identified by, essentially, gut feel and can't be quantified. There seems to be two definitions of the meme and I'd assume that most people around here would refer to the more realistic one (10X better products).
More facts of the 10X meme: not every engineer is necessarily 10X on every project you might assign them and, as you mentioned, a 1X hire could end up being a 10X engineer some day.
Back to the article. An EE team isn't the only way to approach this. We've got at least two where I work:
With enough 10Xs around sometimes you just need to let them fly: give them a little freedom to fix shit, take their fixes seriously and trust them. There's a good chance that if something annoys an engineer enough, they will take their own initiative and fix the shit (sometimes in their own time). This is the approach that my team lead takes - he'll give us the time once we can prove the value of our changes.
Another team lead I chat to a lot logs tech debt the very instant that it is created and allocates time every release to remove tech debt. His tech debt backlog is rapidly approaching zero. The advantage here is that his junior engineers learn the cost of tech debt and learn how to make the correct decisions to fix it as they are involved with removing it. An EE team with 10X elites provides no knowledge sharing.
An EE team might not work everywhere (especially for us: our product could be seen as a collection of products), the most important take-away from that article is that tech debt can be solved with a little responsibility and by granting your employees enough autonomy to figure out how to solve it.
what I found is, that while we are engineers we like technical solutions to our problems. the problem is, that writing software is a team effort where social skills are required to produce good results- a non technical skillset we do net hear very much about at university...
When I got to the first mention of Twitter I hit the back button.
Twitter has a horrible, slow moving, bloated interface. I m facing countless examples of disturbances every day.
Maybe a single platform based on PHP could be better, if Facebook is so usable?
Yesterday I spent half an hour on my mobile just to find out where the "view profile as it would look to someone else" page had been moved.
When tree farming was invented, the initial crop was wonderful, but they soon had to come up with a term meaning "forest death" for the sickly, underdeveloped trees that followed. A monoculture of the same plants quickly strips the soil of essential nutrients and if you cannot identify those nutrients and aggressively resupply them, you will soon find that the trees you wanted to grow and harvest will no longer thrive and somnetimes will no longer survive. A forest can produce healthy trees for generations because of the diversity of plants growing therein. Different plants use different nutrients and some replenish the nutrients used by neighboring plants.
Obviously, software tools are not plants. But I have my suspicions that the diversity of code he describes developed in part for reasons like "different engineers happened to be experienced with different languages, tools, etc" but there may be cases in there where it developed that way because it was the best way to handle it. So when you start standardizing things, you may be killing off things that are critical to the health of the codebase and the company.
I can see value in having someone dedicated to serving the needs of the engineers so they can be more productive. I can see a need to clean things up. But complexity itself often has inherent value that cannot be replaced or improved upon with a simpler solution. Sometimes, simplifying things amounts to throwing the baby out with the bath water. I hope they don't wind up doing that.
I'm having trouble understanding why on earth you would rewrite Scala code to Java. The Scala code runs on the JVM and can interoperate with the Java, so what is the point?
When (say) 5% of your codebase is Scala and the rest is Java, and your engineering team prefers working in Java, those 5% become stumbling blocks. The context switching slows you down and increases the risk of introducing errors.
Also, a tool chain that will support two languages in parallel, while obviously not impossible, is necessarily more complex than only supporting one.
Obviously runs and interops fine, but for less paper cuts when they are modifying it or near to it, it makes sense to refactor it to a common base.
And removing the scala plugin from poms will make the project(s) simpler.
Though as a Scala dev, not the direction I would have chosen, ever.
Really? Not Noah Glass?
Also, no mention of Pivotal Labs role in helping Twitter get rid of the fail whale...
He posits that model and draws many conclusions from it. But how reasonable is? I would expect the improvements in efficiency factor to decay much faster. If everything is done exactly right, there is an optimal efficiency factor. With no ee's there are deviations from that perfect process. As more ee's are added, more deviations are fixed and you get closer to optimal efficiency. These assumptions would mean that efficiency should converge as number of ee's approach infinity. My model suggestion would be
E = (eng - ee) * (1 - b * s^(-ee) )
It seems like the presented formula would say to not devote any resources, but there seems to be some good return on some effort put towards it. Right now, I've just considered it some percentage of my job to work on.
Then Twitter grew to thousands of engineers, many of whom were from Google. Now everything is being converted to enterprise Java in a single, monolithic repo.
Is that an improvement or just bureaucratic standardization and cargo-culting from engineers raised in the google3 monolith?
I wonder how much allowing things drag for so long cost the company? millions in salaries?
Hopefully this blog post's author was unaware of the hundred flowers campaign. In any case it's very helpful of you to point this out so this meme doesn't get passed along by those of us ignorant of this bit of history.
"Let a thousand flowers bloom" has an entirely clear and positive meaning for the vast majority of English speakers.
I wouldn't use it as part of any argument as it will likely lead you to miss important details and cannot lead you to valid conclusions.
It doesn't really make sense to me that a regular phrase like 'final solution', which has been used as a rather awkward translation of 'endlösung', would have gotten the same connotation as the word itself. I must have said "so we end up with a final solution of ..." countless times and I've never noticed a thing.
Language is basically a crappy form of telepathy. You're essentially implanting thoughts in other people's minds. It's crappy because it's really imprecise and error-prone and the minds you're attempting to influence might end up thinking things other than what you're trying to make them think, often because the specific words or phrasing you used brought up associations existing within the target mind.
Sometimes it's impossible to avoid because the association is a personal thing. But sometimes it's widespread. In that case, it's really useful to be aware of them.
I've certainly made my share of bad phrasing in languages I don't speak natively. No reasonable person is going to be offended if you did it because of ignorance, but once pointed out, it's ridiculous to get defensive. Make a note of it and use this knowledge to improve.
Oh yes, someone is callous unless she learns the sensitivities of every person and group in the world, and then avoids them all whenever posting anything on the web.
Or, you know, we could just be more forgiving and understanding of each other.
It's an blog post dude, not an insensitive advertising slogan for a new foreign market, get over yourself. Assume he didn't know any better and move on.
Being upset about using this phrase is like an army of tumblr SJWs being annoyed when someone says gypped. Not everybody knows the complete cultural and racial history of every innocent common phrase used today, even if they come from questionable historical pasts.
Here's what I see: jordigh says he wishes people would learn about the hundred flowers campaign so they won't make such bad allusions by accident. coldtea counters this saying that it's not actually bad. seiji agrees with coldtea and attempts to offer further support by saying that people don't learn about this stuff anyway.
Not so?
Ain't that hard.
Having thought about it, it doesn't really make sense, since I'd apologize for hitting someone even if I didn't mean to, and even if I couldn't be reasonably expected to hit them.
But when the issue is offense, you don't want to apologize, because you think that they think that you meant to cause offense, and apologizing will reinforce in their mind the idea that you really did mean to do it, or at least in your mind it will reinforce that idea in their mind. And you don't want to reinforce that idea in their mind, partly because it makes you look bad, and partly because the idea is simply wrong. So you don't want to apologize.
(Note well: "you" here means "me, too, given the right circumstances".)
Worse: That may really be going on in the other person's mind. They may really think (or at least suspect) that you meant to cause offense, and therefore are determined that you must apologize as penance or proof of repentance or some such. "The good of society demands that I force this person to apologize!"
So maybe the best answer is to say something like: "I did not intend or wish to give offense. Apparently I did; I apologize for doing so." Because the fact is, the offense was real. Apologizing for it is just good manners. But apologize without agreeing that you had intent.
This presumes that you didn't have intent. If you intended to offend, that's a whole different situation. It's worthy of an apology, not just for the offense, but for the intent.
I haven't seen any "you're an asshole" type comments in here--more "aw, c'mon, man." (I frown more at whatever instructors the author had in 20th century history a lot more than the author himself. This is something that's not that obscure, but you gotta know it exists to know about it.)
If memory of my history class serves, the quote of letting flowers bloom comes from Mao, who told people he wanted them to open up and share their political differences (ostensibly so that some kind of political dialog could follow), but with the ulterior motive of merely outing his political enemies in order to do terrible things to them.
It's a shame to be discussing this instead of the article, though, since I am a huge Peter Seibel fan!
Nobody much considers "let a hundrend flowers bloom" (as a phrase) equivalent to the "final solution", not even in China. This includes people who know all about Mao, the cultural revolution, the power plays and the executions that happened at the time.
It's actually a stock phrase in journalism and writing in general in lots of European countries (I know that's the case of its use in at least 4 countries/languages).
This looks more like someone in the US, several times removed from the events (by ethnicity, nation, time, etc), delivering a paternalistic lecture on foreign history.
Examples of casual use in both US and UK:
http://www.economist.com/news/britain/21581984-inner-city-sc...
http://www.washingtonpost.com/archive/politics/1986/11/03/we...
http://www.theguardian.com/politics/2015/sep/16/national-ant...
But in any case, "Let a thousand flowers bloom, then rip 999 of them out" is much more evocative of those events than the casual uses you cite.
It's reasonable to infer offense from that statement.
That's correct, and I wouldn't mind if it was just that.
But OP's "I wish people would stop" etc and calling it "dumb and insensitive titles" goes a few steps beyond "pointing out that this phrase might carry extra meaning".
Likewise, I think it's disingenuous for readers to begin their indignation at every corner they find someone who is ignorant of the baggage a phrase or word might have with other people. I sometimes wonder if these kinds of oversensitive people don't go about putting a list together of all the inventions and things furthered by disgraced people because partaking in such things would give such detestable things their imprimatur.
So and so was a philanderer, this other one was an opportunist, that one was a bigot, this one was sexist, that one a hedonist, an atheist, a Christian, short, stinky, tall, too cultured, not my type, too much my type...
I don't take offence, but may suffer injuries related to involuntary eye rolling. It's an overused phrase and I'll assume the author is a hack and stop reading. Sort of like anything with "cyber" in the title published after 1993.
Now you aren't wrong. But I also don't think it's fair to call dumb and insensitive. It's not dumb because it's a perfectly reasonable and useful analogy. And I don't think it's fair to call it insensitive because the author almost certainly didn't know the 60 year old foreign history from which it is derived. That's not someone being callous or insensitive.
It's just good old fashioned ignorance. But ignorance that's kind of ok and fine. Like, it's just totally ok for someone not to know the origin of the phrase.