A dev's thoughts on developer productivity
about.sourcegraph.com
about.sourcegraph.com
The bad reputation of developer productivity metrics comes from the misguided assumption that developers should be measured. The better approach is to treat developers as customers of the management team / engineering enablement team /etc. In that sense, developer productivity is actually a measurement of management effectiveness / organizational health.
Once you build your developer productivity approach on this basis, what needs to be measured becomes much clearer - for example: interruptions caused by meetings, performance of local tooling like local builds, latency of CI, number of on-call pages triggering outside business hours, etc.
The right set of metrics depends on your team and can be sourced from surveys and just talking to people about what's painful on a daily basis. As a quick and dirty solution, I'd even recommend piping Github webhooks and other events to a product analytics tool like Amplitude or Mixpanel. You'd be surprised how fast you can understand things like build or CI latency by using a classic funnel visualization.
A lot of great engineering teams are migrating to this approach, especially when they have a dedicated platform / enablement / productivity engineering team.
But those metrics don't really help management decide how many hours developers must be required to sit at their desks per day, how much unpaid overtime to demand from them, or how many years to wait before giving them a tiny raise! /s
What you are describing is "treat developer productivity as a supply side problem." developers continually demand resources to improve their productivity.
However, there's two issues:
#1: Developers don't necessarily know how to be productive developers.
#2: Developers might not be motivated to improve their productivity.
Hence, it's not necessarily an efficient market.
I find that you need to control for market inefficiencies by:
- Control for #2 by having a "tech lead" or senior engineer be directly responsible for their developer's performance. Whether that's a 2 parent leadership team (people manager + tech leader) or otherwise, developers must have direct oversight of their personal productivity.
- Have an appropriate incentives in place for developers to improve. A couple of places (IBM) actually have an excellent infrastructure for providing productivity. But no incentive.
Tech leads in most engineering firms don't have that kind of control. That's why OP mentioned "manager". You can't make someone responsible for stuff they don't control. Even if a manager "tasks" a lead with this they usually don't in fact own it. The paradigm you're asking for requires a people manager to be leveled the same as their tech lead so that they share responsibility for outcomes. Most places don't work that way.
> Have an appropriate incentives in place for developers to improve. A couple of places (IBM) actually have an excellent infrastructure for providing productivity. But no incentive.
OPs point is that developers are a reflection of their environment more than they are of their own knowledge. There's a DevOps study from DORA that covers this I think, as well.
That's a mistake by most engineering firms.
Direct technical leadership is extremely important. Weekly direct feedback and performance evaluation on technical tasks. By a technical leader who is directly responsible for getting this person improved.
I think, the vast majority of the status quo re: "managers" in the software industry is absurd rubbish.
I use the army/medicine analogy.
In the military/medicine, a junior doctor/2nd lieutenant has every aspect of their work overseen and audited by an expert who can do their job better than they can.
In the software industry we often skip this step.
Most software is cut price rubbish. Engineers who enjoy leadership and mentoring other engineers cost $$$. It's not cost effective, so bad management ensues.
> OPs point is that developers are a reflection of their environment
Definitely agree. I'd say my points are additive, they're not an alternative to OP's.
"#1: Developers don't necessarily know how to be productive developers.
#2: Developers might not be motivated to improve their productivity.
Hence, it's not necessarily an efficient market."
Has been my experience lately in thinking about this problem. Those who are productive generally know how to be productive and _want_ to improve their productivity. Many have the motivation but don't know how, and many others are hampered by external factors that limit motivation, or don't care.
Says a blog post from a CTO who sells a product to track software productivity.
For what it's worth, I don't think that's accurate. I've worked closely with the team that builds Code Insights, and at pretty much every point they've been vehemently opposed to exposing any metrics that could ever be used to track/monitor devs.
On the contrary I've actually heard most use cases of it in companies are actually devs themselves showing leadership/management clear graphs of "look, code quality here is getting worse, we need more time in sprints to improve this"
Anyway, just my take, not commenting in any official capacity.
Quality of user stories and product cycle would be interesting to see as it has a direct impact on dev quality (ie how much do they have to guess versus cleanly mapped out).
It's pretty cool and I see where you are going with it.
We do have a feature (https://about.sourcegraph.com/code-insights) built around code search that lets devs define their own metrics around things like progress of big migrations and anti-patterns + code smells. This tweet is a good summary of my philosophy around code-related metrics: https://twitter.com/beyang/status/1524259425451577344.
Insights from treating code as data are best discovered by the people who know the dataset through and through—i.e., developers. Another project we've released in this vein is https://codestat.dev, built on top of Comby (incidentally also on the HN front page right now), which was created by a member of our team, Rijnard, as part of his PhD thesis on automatic program transformation.
We're developers by trade and by heart, and are motivated by making tools and mental models that devs actually find useful and that boost the creative spark of coding. If there's something we can improve about our product or message to further this mission, I'd love to hear your feedback!
Most orgs kill productivity and creativity with lack of compensation, treating them like a code farm, or mediocre peers.
Devs choose to be productive in different environments, most of the time it’s simple, step aside and let the dev do more than code. Talk to a client (yes! It’s not as crazy as it sounds). Show appreciation with bonuses not a seasonal holiday themed ham.
I think your headed in the right direction!
"Sourcegraph is a web-based code search and navigation tool for dev teams. Search, navigate, and review code."
On the inner loop, and progress being a vector sum: one thing I find crucial is knowing where I have to go. That translates to two mantra: rush to the finish line, then rebuild it if needed.
When embarking on a project I know roughly what stages are needed. Maybe three dependencies, one more to bring them together, one more to deliver a subsequent result, and one outer one to present the whole code. Example: a web scraper to get weather data, a thermometer polling daemon, a database to store the two, a scheduler to run the first two and store it in the database, and a module to perform high level queries on past data and forecast trends for the future.
It’s really easy to deep dive on any one of those components. It’s not until I get to the final one that I realize how little I really know about problem. I’ll explore it a little, then work on something else for a week, then come back and know exactly how to retackle the problem from start to finish. It’s the revisiting that gives me insight into what to do, and it was the rush to build the prototype that gave me the raw materials needed to have that insight.
All of this requires at least one whole day of concentration, then three half days of tinkering, then a final week of rewriting and polishing and responding to code review.
So much of this process is subject to external forces that impact my productivity. I am grateful to this article for solidifying them in writing.
If instead you rush, you get a longer average free-path and may gather enough information from the results to decide a direction.
The situation is different if you already know where you are going before you start.
And I agree, it's a very good advice and not always so easy to follow. You often need to cut corners and ignore stuff for later to connect the system ASAP. It's mostly through experience that you learn what can be ignored and what should not be ignored.
When I focus on building a really flaky version of a thing as quickly as possible, then I quickly land on all the edge cases and more-complex-than-I-thought areas I need to deal with.
Sometimes I can refactor my flaky code to deal with those, but more often than not it's easier and faster to just throw it away and start again with my new-and-improved knowledge of what needs to be done.
The challenge is then (a) working out how far to go down the quick-but-flaky path before starting over, (b) working out whether to keep sections of that flaky code or rewrite it, and (c) trying to not get scrum masters over-excited that I built a barely-working thing in 1 day when the working-well thing is still a week away
One way to do address this is - to be explicit about the flakiness of the work by marking it as a spike story[1]
While this is less than ideal, it is a formal way to get the message across.
[1] https://www.agile-academy.com/en/agile-dictionary/spike/
Software defines (or will defines) a company - what it can do, how it does it, how it reacts. If you cannot read the code you don't know what the company does. If you don't write the code, you cannot define the company.
Defining the activities of an organisation is management. All the people called managers are at best coaches, giving a towel and water in the corner.
And so productivity is measured by organisation chnage - otherwise you are trying to measure typing speed.
(this is not to say we do not want to look at the tools we use to produce software, but this should be focused on CI-like throughput (ie Speed vs stability) not "correctness"
Some activities/behaviours may still be injected at the developers’ level, but only if it does not contradict or impair what was defined as the main activity.
So while, yes, developers may evolve in those roles where they will be in position to learn and understand what the company should do, and communicate it, and steer people to implement it (in code and other things), they won’t have much brain power left to read or write code. They will specialize in their new role and rely on other specialists.
the rest of the 20% matters - "culture", core competencies and so on.
but strip out much of the busy work - have a "scalable company in a box" and just ask the founders to focus on the bits that really matter, and i think the brain space will be found
I understand this bias/will in the software industry to abstract things out of the physical/human world, but in the long run, what makes a company/business run is the people running it, not the tools/gears.
If a company can be just founders alone, fine, why not, but it's not sustainable in the long run. If you need to hire and lead people for that purpose among others, software/APIs/implementations are a competitive advantage, but not what makes the business heart beat.
while being a developer myself and thinking that 70-80% of all management decisions are well undercooked, i believe that there is so much more in here that is not what developers do or want to do. Yes, we write how software and - in the long end - companies work, but to actually specify this there are so many factors to be included.
What is the market doing? How do the customers react to certain approaches? What are our ressources? How competent are the users in our company with using processes/software/communcation?
IF we are the new managers, we should try and raise our heads from time to time and look at the whole company. I've seen so many developers producing good software, but software that is not working with and in reality (being a running business with people and real life problems).
Let's divide management up into administrative / supervision and leadership / creation and mentor / culture
the first part os getting replaced no matter what. The last part - mentorship etc, well I am not sure that is done well almost anywhere.
And that's probably 80% of current management work
The real nitty gritty is leadership / creativity.
Look at some of the best in OSS - let's try Linus, or Van Rossum. Yes code is produced. Daily. But also politics is navigated, usually
The good things here are that the politics is open because the only means of communication is open(ie email). If those with commit bits could meet in smokey back rooms they would. it's easier.
not sure where i am going but software gives huge speed and flexibility advantages that not to replace most decision making in your company with software is like refusing armoured cars because horses are brave.
1. Work on simultaneous tasks at once. This incurs a context-switching penalty; or
2. Get distracted by HN, social media, reddit or whatever. This has the same context-switching penalty and you can end up spending way more time on that distraction than the development loop actually needed.
I've worked on a code base where it took 3 minutes minimum to run a unit test. Some adjust to this mindset by doing more in a single test but even in the most optimal fashion it's not as good as quick turnaround.
Compiler errors are strictly superior to runtime errors. It's as simple as that.
Compare that to an environment I was in a few years back where I was lucky to get two builds a day. Sometimes went weeks without working code because people kept checking in busted stuff. I wound up quitting (for that, and other reasons).
It is amazing how companies will pay high salaries for engineers, and then refuse to buy them top of the line equipment, too. I remember fighting for SSDs, when the time savings alone would have paid for itself in a couple of days. Crazy.
In the .NET CMS Solution, when I make a code change (view changes are easier), if the change restarts the application pool, it can take up to 2 minutes to see my change - worse in some solutions. (I work to support some existing solutions that weren't written by us but only have a certain amount of hours to accomplish, so doing the "best" thing isn't always an option). So click reload and then I'm off to another project and/or distraction and may not be back to check the change in quite a while.
The vuejs project on the other hand. I make a change to the backend, reload the app - in under 2 seconds and then re-run the API method. Or even better, if I make a change to the frontend, I hit save on the file and the screen is updated by the time my eyes have a chance to move to the other monitor! It's so refreshing.
https://littlegreenviper.com/various/concrete-galoshes/
https://littlegreenviper.com/miscellany/forensic-design-docu...
https://littlegreenviper.com/various/evolutionary-design-spe...
https://littlegreenviper.com/miscellany/thats-not-what-ships...
https://littlegreenviper.com/various/testing-harness-vs-unit...
https://littlegreenviper.com/miscellany/leaving-a-legacy/
https://littlegreenviper.com/miscellany/risky-business/
https://littlegreenviper.com/various/the-road-most-traveled-...
"Flow state is that state of focus and productivity that you attain when you’re feeling inspired and motivated."
...which makes perfect sense as two of the worst-offenders were places where I was expected to put out lots of code requiring flow-state, while simultaneously being immediately and constantly available to instant messages and email-used-like-instant messaging.
The scary part is, the ability to focus and produce like I could the first years I was programming never came back. Now, I'm working at a place and in a role where my employers seem happy with what I'm doing, but I'm not happy with the rate I (can't) produce at.
The problem here is that if I'm not being productive (I'm teaching), and the other devs are not being productive (they don't know the domain), then nobody is productive.
You'd think that'd level out eventually as the other devs grow, but on short runways, there really isn't time for that.
However, we can't hire devs who know the domain because a) they're not looking for jobs and b) the ones who are we can't afford.
So it's just me, teaching other devs, when I could/should be pounding out code so we can get a few more sales, which will justify the next round.
Unintuitively, it probably didn't make sense to hire anyone else, but that feels weird because then it's just me in a dark room churning out code nobody's reviewed, and there really is plenty of work to do, it's just all work only I seem to be able to do.
I assume this is standard for early stage startups who haven't yet found their P/M fit, but it is odd.
There's never a clear blueprint but a couple of ways to tackle various flavors of this problem:
* Pair a good engineer with a non-engineer domain expert and let them teach each other as they ideate and build.
* Scope down the projects instead of scaling up the team. Without P/M fit, the team might need to focus their energy and get initial traction solving one aspect of the customer problem.
* Scrap any tech stack complexity and go full hackathon-mode until something clicks.
Patrick McKenzie has a good take on this along the lines of "teach someone to fish" (s/fish/code search/): https://mobile.twitter.com/patio11/status/108611151367121715...
We've tried to make it easier to turn organic code explorations into interactive docs that can be read many times over with a notebook-like feature. Here's a notebook that bobheadxi wrote up after he noticed he was explaining Sourcegraph's authz model for the billionth time: https://sourcegraph.com/notebooks/Tm90ZWJvb2s6NTc0
"Perhaps as a consequence, we’re swimming in acronyms, magic metrics, and methodologies, but not a lot of rigorous systems thinking."
I've said this before, but I want more developers to write honestly about their day-to-day experience in the trenches. The real life experience is often something quite different than "rigorous systems thinking." The whole industry would benefit if we had more honest accounts of the kind of bad management, and process chaos, that leads to high levels of tech debt. Many projects fail outright, but we don't get many good postmortems on the absolute failures. Of course, most of us sign NDAs when we join a company, so we need to step carefully if we later decide to pull back the curtains and reveal "This is why that project was such a disaster." Sometimes we have to change people's names, so we can speak freely. But again, I think the industry would benefit from this. It's what I was aiming for in my book How To Destroy A Tech Startup In Three Easy Steps, and I think the reason that book has become popular is because people appreciate the raw honesty. I do think we benefit from good essays that remind us how "rigorous systems thinking" might actually lead to better development processes, but such a theory of ideal development needs to balanced by the many factors that lead to pure chaos in many projects.
Ive got to say, what I pictured would happen from reading books and blogs and even getting a fucking degree in CS was so different from what actually happens in the daily lives of software developers. Im not saying its always negative or positive but just so different.
If only as a service to past me, It would be hugely beneficial if we had more raw accounts of what it’s really like.
Your options are to think nobody understand you and everyone is working against you, or to take ownership and defend your time accordingly. Which is incredibly easy to say and very difficult to achieve.
I worked at a place once that had one day a week where meetings were banned. Across the entire company. I've since realized that it wasn't the "no meetings" aspect that was the value, it was the co-ordinated scheduling of time for deep work. The biggest challenge I have today with defending my time is co-ordinating it with other people. At a sufficient scale everyone has a different block of time you're trying to defend, and when you're trying to get 4 people together to have a synchronous chat it almost inevitably clashes with someone's block. Having a mandated "everyone has at least one very long deep work block every Thursday" was a powerful way to preserve _at least_ one large period of focus each week. You're still free to schedule others too.
I'm not sure if you meant it in this way, but "defend your time" is a phrasing that implies an adversarial relationship between your time and those that are supposed to be facilitating you to do your best work, like a manager. I believe a manger is someone who removes distractions, buffers you from interruptions, and facilitates frictionless coordination with others. Anything less is just making the hard job harder.
> What good is “change failure rate” if you can’t even jump-to-def across your code?
Exactly. Organizations often focus on the externally visible factors without considering the day-to-day of a developer productivity. If only we spent more time to refactor/maintain and general tooling instead of more status updates and unnecessary processes, imagine the productivity we could achieve..
At Google we had this simple problem yet it halted our entire team's productivity: IntelliJ started being insanely slow indexing files and auto completing stuff. It was a mixture of generated code and the monorepo that made it take tens of seconds for any kind of autocompletion. It's been there for years but there wasn't much priority to fix it, so things kept going (and probably still keep going) like this...
On the other hand, if the industry truly evolved, it would probably up the barrier to entry to a level a lot of developers would be out of a job.
Is a college degree absolutely necessary? Probably not (but extremely valuable, nonetheless). But bootcamps are completely worthless. At best they can operate as a screen for the already capable, but they certainly don't market themselves as such.
There is a team that focuses on fixing these smaller issues, in addition to a larger dev productivity team that works on the more gnarly problems like “building takes too long”. Importantly, the team DOES have an OKR for fixing paper cuts, and gets kudos for doing so.
I'd like to balance on the importance of the flow state.
Maybe it's just me, but the more I learn about how I work, the more I force myself to interrupt myself regularly.
I find it is very easy to go deep down chasing a bug only to realize later that different paths are available.
Being able to work deep is important, but I find that pausing regularly to reflect on the work done and the target is equally important.
When I work in pomodoros I can take short breaks every 30 mins and they seem to help - precisely due to the stepping back and rethinking you mentioned.
Other people have commented to me how context switching is not so hard for them.
I think there is a “deep flow” state, but it’s not actually so productive - you just feel focused and do a lot of work, but it might not actually be in the most efficient direction.
So I feel there’s this trade off between “flow” and “managing yourself”. I guess hackers of repute just flow so much they compensate for their lack of self management.
I work in a large organisation where some of the issues mentioned are very relevant, especially the 'flow state" issue.
We have a mix of on-site and remote workers (myself being a remote worker), and because Slack/Teams is our primary method of communication with colleagues, and I operate in a tech lead role, it becomes difficult to achieve flow state on some tasks.
I ended up creating an Apple shortcut that automatically setup the computer for operating in a flow state (called Superbam - https://www.icloud.com/shortcuts/5c0c729476584e11b3b12744893...). It helped to some extent.
I think a lot of organisations are clueless about flow state and why they should give developers the chance to reach it.
I think one key under-explored opportunity for improving productivity involves bringing the outer loop closer to the inner loop. The main reason we have a distinction between these loops is everything we need to do in the inner loop generally happens fast enough to keep us in flow state, while everything in the outer loop is usually slow enough to break flow state.
In the graphic illustrating the inner loop vs outer loop in the article, only the Plan, Review, Measure, React, steps have intrinsic latency floors based on product/market/human/organizational constraints. My hypothesis is that the other three, Author, Test, Deploy, are constrained in latency only by technology, and can be made fast enough keep us in flow state given the right technology, bringing them into the inner loop.
Disclaimer: My ulterior motive in writing about this is I've been building a product to do exactly what I described above, for client-side rendered React apps: https://reflame.app/. But I would honestly love to see more folks explore opportunities here, since I want to be able to build all kinds of software this way, not just the kinds I have the bandwidth/skills to work on.
Abstract: The Personal Software Process (PSP) is a structured software development process that is designed to help software engineers better understand and improve their performance by bringing discipline to the way they develop software and tracking their predicted and actual development of the code. It clearly shows developers how to manage the quality of their products, how to make a sound plan, and how to make commitments. It also offers them the data to justify their plans. They can evaluate their work and suggest improvement direction by analyzing and reviewing development time, defects, and size data.
There is also a corresponding Team Software Process (https://en.wikipedia.org/wiki/Team_software_process).
Instead of coming up with more ad-hoc methods of measuring "Developer Productivity" i think it would be better if people build upon this. It is a framework to define a specific tuning process and hence easily adapted to any methodology (Agile or otherwise). The reason i like the PSP/TSP is that it is solely for the Developer's benefit (no Management need be involved) where i the Developer measure my performance on various metrics (which i can define) and improve them as i need to i.e. it is a proper Self-Feedback loop.
In general, team productivity will fall after the novelty of a problem wanes post launch, and faster with bad management policy. Micromanagement proponents (apps, nuisance ticket notices and meetings etc.) can increase output for low-skill support tasks, but ultimately fails in a Sr. developer sense. In general, I find team members naturally volunteer to take responsibility for a specific area, and if it works out well... i.e. on schedule... The criterion of “productive” which matters are met. It can also be brutal on individuals that suck at their job, as everyone in the room already knows who messed up.
;-)What about Slack?
What about even email?
Even if we worked for some magically company that doesn’t have meetings, if your expected to always be on Slack and reply immediately - you’ll never achieve no distractions.
1. minimal distractions at 2. scheduled times that are 3. contiguous
I'd go so far as to say that there are times where it's better for a single dev who is stuck on a project to keep his mouth shut for an hour or two than blasting a slack channel and blowing everyone else's concentration.
Nothing destroys productivity quicker than flow disruption: eg. lots of "really quick" meetings, bonus points if they are spread about 45 minutes apart and take no longer than 15-20 minutes.
What does screw me is loud noise, be it shouty workmates or my absolute utter nemesis, music piped into the office.
It was fun in the 90s when everyone was creating their own.
I note some particular passages:
> Perhaps as a consequence, we’re swimming in acronyms, magic metrics, and methodologies, but not a lot of rigorous systems thinking.
Performance along sustained, abstract effort is far from rigor. Measuring it causes a wave collapse so to speak.
> Devs are systems thinkers. Our job is to model and build systems, and we often draw out diagrams and schematics to illustrate how those systems work. But when it comes to our own work, we are letting others draw the pictures—and they’ve done a poor job.
This is systemic. The trenches don't build themselves. And we don't get to decide when to charge at the same time.
> that feeling of flow can be quite the gateway drug. But many get stuck in the second graph when they code professionally, and that is a great source of developer misery and lost productivity.
No comment here, just interesting.
> productivity
Is a loaded term.
> Software development is at its core an innovative endeavor.
Now we're cooking.
> The goal is to produce new and useful knowledge.
If only SaaS would stop playing mind games with each other.
> but iteration frequency. We might call this unit of quantity developer hertz
No fuck off.
> You can draw this out in a picture where you’ve broken down a project into different deliverables. The finished product is a pyramid of sorts, where the blocks at the bottom have to be laid before the ones on top:
I choose to see software dev as a cyberpunk vertical foundation where you can scale many foundations wide, and scale many stories high, and at some point you might lean-to on high-rises to bespoke a business vertical. This of course decreases flex, but at least we have the blueprints.
> The two developers have the advantage that they can build separate blocks in parallel, but their parallelism is constrained by serial dependencies (some blocks must be laid before others), the friction of onboarding to unfamiliar parts of the code, and communication overhead.
Mmmm. We are getting closer to my thesis
> It is to our benefit, and to the benefit of our industry and society at large, to advance mental models that reflect the reality of our work and honor its essential creativity
This is why we are inconclusive, not that this piece was an oracle of The Goal, but more of a stimulant to sell more books, or transfer your thoughts to your organization.
IMO, I think devs need an internal PR shield, like the DPS tank sponge, to soak critics, and communicate technical limitations, and coerce a system that bends like a tree branch. Too far and you'll snap into firewood, too little and you will only be one vertical.
The PR team paints many directions, and belays the authoritative button pushers who understand only abstraction.
Truly, developer salesmanship.
Now available at your local MANGA shop.