Prolific Engineers Take Small Bites
blog.gitprime.com
blog.gitprime.com
This is also the primary reason I have switched to Jupyter/R Notebooks for my blog posts: if I make ridiculous claims, people can check my work as evidence. This post doesn't even provide any quantifiable metrics, just "we analyzed millions of commits."
In order to dig into enterprise data, we sort of need to have a ToS that only allows us to talking abstractly about aggregate data.
Any thoughts on how we might navigate that?
I'm not saying that you are necessarily wrong but I do wonder if making an article like this more like a formal publication will actually make it better or worse at it's real goal which is presumably to sell more.
More importantly, calling something a marketing article does not immunize it from critical analysis, and especially not on a forum such as HN.
I wondered this also, when the popups in the lower right kept presenting a chat box with "Can we help you with something?"(from gitprime). Do they want to sell me something, or do they want me to read their publication?
It's a different type of content, which has a different kind of value. Are people trusting the claims the way they'd trust journalism from a known good media organization (or a paper from a known good research organization)?
(I'm genuinely confused about how this person is pretending to be a journalist. It doesn't look like journalism at all to me, starting from the part where it's on some startup's blog. What am I missing?)
Except what we really have is a lack of data backing up the fancy non-informative charts (that clearly in no way shape or form correspond to the so-called millions of data points).
Doing less is harmful to your own reputation at best, but can be harmful to whoever believes in the things you got wrong.
Now that applies to mistakes. But for those who intentionally write and publish misinformation - I hope there's a special circle in hell waiting for them.
Lots of plausible deniability, here.
This blog post is pretending to be true.
I want to contrast it with this: another startup(ish) blog post about their own data
https://www.backblaze.com/blog/hard-drive-reliability-stats-...
That is a wonderful, useful, trustworthy source of interesting data.
The post source seems like made-up marketing fluff pretending to be real information. I'm looking at graphs I entirely believe were made up, I have nothing to support the conclusions, and I'm very concerned that a less wary reader would think it's true.
If it had said something like tools of data journalism I wouldn't have been inclined to reply. A small difference, but enough for me.
Actually, you can say anything you want with evidence to back up the opposite of it ;)
I have a hate-love relationship with journalists. It's both incredibly surprising and incredibly frustrating to discover what they end up publishing after an interview.
Basically, start a top-down refactor. When you find the bottom you stop, revert your code, and refactor from the bottom up.
You probably won't find the actual bottom, but in my experience the moment when I was about to give up and call it quits was usually just before I reached the bottom. Taking a break, and starting over from the spot where I stopped, usually the solution became obvious, and far less invasive than I expected.
You start refactoring at a high level -- essentially set out to rewrite the whole chunk of code in which your refactoring work lies. And focus on the most important until you get to the inner-most structure.
Then when you are to the "sum(a,b)" level you start over with the original code base and refactor bottom up. With the knowledge of how it works all the way up to the top? Re-creating your original refactorings?
It seems interesting. Like tackling a math problem or puzzle from two directions. Are there certain situations in which it is most effective? Architectures? Paradigms?
When the architecture problem has gotten away from you, or only becomes apparent late in the project, it's hard to see the trees for the forest. The only way to decompose the problem is to start somewhere and see what happens. With a top down rewrite the number of bits in flux becomes overwhelming, which is the anxiety I picked up from the previous commenter.
Mikado is just an exploratory development trick to help you find a way to make the change with refactoring. You try rewriting pieces until the number of concerns starts to multiply, you keep going essentially until your brain starts telling you, "this is nuts, you should stop", and you take that detail that broke the camel's back and do just that part, and build up and out from there.
For R Notebooks (which are HTML files), you can use GitHub Pages, which is super easy to set up nowadays, as the static files can now be located in the master branch.
A few hosts that currently support that are PythonAnywhere and as of this month, Azure [1].
This is embarrassing from a YC co.
Source metrics are the new panopticon of Software is Eating the World.
As for their size measure, they are vague, but the comment about 'normal' commits raises the question of whether they have crafted a metric for which the linearity is built-in by definition.
You’re right: this post is a narrative about product development, and a strong correlation we found between two variables across 20 Million+ commits that we thought was fascinating and supports general 'kitchen logic' around best practices. The axes are not labeled, but if you like we can set you up with a demo account and walk through your data with you.
One other note: the typical common use case for the product has been stakeholder management; something we have doubled down on in product development. Any specific critique about how we can improve most welcome!
1. Data is always necessary to back up claims like these. Almost _any_ kind of quantitative data will do - just a simple average! just give me something I can replicate on other data sets. Similarly, just saying "we made up these variables as a combination of these other variables and called them Flavorfulness and Musicality, look how nicely our circles line up!" is not useful to anyone - a formula or methodology would be.
2. As a software engineer, I came to this post expecting to find a way to improve myself. I invested time in reading it because I expected a payoff in the form of, for example, a metric I could apply to my own work. I suspect that the author of this post intentionally gave the impression that the post contained tools like this. I didn't find anything remotely like that.
3. More generally, this post does not show signs of being written with the reader in mind. You have not given me any new information that I can apply; instead you have given me a marketing pitch (we did this data analysis, we promise! Don't you want us to work with you?) dressed up like information.
Seems like you've given this some thought on how this should be done right; would love to include them in our product development discussions.
Really? Rather than going back and labeling you axes you are converting this into a sales call???
Similarly, the offer here is to take a deeper look at what we're building and a (quite genuine) offer to incorporate any suggestions you might have into our roadmap.
Your post gives prescriptive conclusions not supported by the data presented. Of the data that you did analyze, we're given so few numbers actually published that everyone is forced to question the fundamental methodology, even that is not clearly stated. The specific critique is to make well-founded claims.
On the other hand, the irony of being told what to do by a tool coded by some 'out-of-touch know-it-all tech people' is not lost on me - some of my work has surely been used to inflict this sort of micromanagement on others.
https://blog.gitprime.com/impact-a-better-way-to-measure-cod...
What I found is:
The amount of code in the change
What percentage of the work is edits to old code
The surface area of the change (think ‘number of edit locations’)
The number of files affected
The severity of changes when old code is modified
How this change compares to others from the project history
Based on the vague description above, nothing in "impact" is directly related to the actual end user/paying customer experience or a reasonable proxy such as systematic end user testing by a QA team.
This lack of a direct relationship to the desired end result is the same problem that lines of code (loc or LoC) and many other metrics of software engineer output have.
The "impact" metric, whatever it precisely is, looks suspiciously like it would naturally be positively correlated with a large number of commits/high frequency of commits.
Also the plot is labeled with "volume" on the horizontal axis and not the mysterious "impact" metric. The text implies this horizontal axis is the "impact" metric. Why is the horizontal axis not labeled impact?
Even more peculiar, "impact" is claimed to measure cognitive load:
Impact attempts to answer the question: “Roughly how much cognitive load did the engineer carry when implementing these changes?”
A good engineer will attempt to find a low or no cognitive load solution to a problem! In general this will be faster and less error prone and cheaper! Reinventing the wheel has a very high cognitive load.
If my job were being judged by this metric, I'd sure be thinking a lot about that.
This measurement also seems to favor overengineering, which is one of the biggest things I've learned to stay away from. When I was early on in my career, I thought I could implement anything myself, and so I often did. It wasn't until years later that I understand the implications of maintaining these implementations.
A true measure of impact to me would be a ratio of the most concise (but still understandable) amount of code to the amount that it solves the requirement it's built for. It's impossible to measure that without the context of the problem you are trying to solve.
Yes, a lot of (although far from all!) good coders have the habit of taking small bites. That doesn't mean though that by taking smaller bites you are (or are becoming) a good coder. I've had the past displeasure of dealing with code that hit all the superficial checkmarks (small commits, unit-tested, peer-reviewed, you name it...) yet completely stunk.
Both times I tried to get information out of their site they found a way to interrupt me.
Then I leave without reading the article.
"Do you use an Ad blocker? I do, and didnt have that experience."
First of all, sometimes it's most useful to take a step back and think about problems in a wider context. Often the biggest impact is from the code you don't write. There's really no way those kinds of contributions come out in any kind of metrics, but the team will sure remember them.
More commonly, I prefer to commit often to aid my dev process but then rebase those commits into more cohesive units before merging. How chunky to make those final commits is somewhat a matter of taste, and probably shaped by the domain one is working in. Personally I like the finest grain possible without any build breakage, that way git-bisect still works, but hopefully there's still enough granularity to help with trickier merge/rebase scenarios around dependency upgrades and/or db migrations. But pardoning the digression, my point is the output seen by the rest of the team may not represent the original frequency of commits.
(My issue tracker isn't coupled to my repo, and bisect is mostly useless for me because my builds take so long. So like you said, depends on the codebase.)
Their methodology fails to account for so many things, commit squashing feature branches, for example. A fundamentally flawed "study".
If you're using a code review tool well (if you are more experienced with it) then you'll automatically be making lots of small commits vs people that may not be well versed in working that way.
If you're working with large or prolific teams then you'll be committing all the time with eg feature toggles to ensure everyone is as close to the current code with their changes as possible.
So making general claims about more commits without taking into consideration the ways that metric is bent and changes with tools and processes and different experiences seems to be dangerous.
Impact as a metric is fine, but you have to show what it is and how you arrived at that conclusion, and have verified that against a test set of data otherwise it's turtles all the way down.
This leads to bad coders not being able to commit as frequently as good coders.
However, this seems like a very, very loose heuristic.
People who have worked in code review shops tend to stage their commits, i.e. do a bunch of work but commit it with git commit -p.
Article also doesn't look at deployment frequency, and 'merge' appears only once. We don't know if these 'high impact' devs are in their own branch for 6 weeks cooking up the PR from hell. That's one way to have a high impact.
(Although I'm sure that's part of the secret sauce).
Essentially what I'm getting at is that peer review from team members is the best metric for what is good and isn't.
One thing that strikes me about the majority of the submissions, as funny as they are, is that they mostly boil down to "so-and-so didn't know that such-and-such feature existed, so wrote reams of code to implement that feature in a complex way". It also strikes me that just this article's sort of analysis of "prolific" (aka "good") engineers/programmers drives this same sort of behavior. If every developer is supposed to be committing code all day, every day, there's no time left over to read the product documentation, try out a new feature, review a reference implementation, read a blog post: to be "good", you must be spending as much time as possible _typing_, because that's what you're paid to do. This (ubiquitous) management mentality is how we end up with roll-your-own crypto, or five competing Javascript frameworks, parsing using regular expressions... it's not so much that what they did was wrong - and trust me, if it works, it won't be removed - it's that it's pointless.
If you're doing your work in small, sharp bursts, you actually end up with more time in between work for talking with other devs, reading up on technology, etc.
A lot of DailyWTF posts are about people not knowing a function existed and then reimplementing it horribly. Many of them include a note like "the other 10,000 lines were similar".
Ignorance is not embarrassing, it's the default state when all of our tools grow and change daily. But a lot of the worst wtf moments are clearly cases where something wasn't sanity checked by anyone until it was presented as a finished product - sometimes too late to keep out of production. That's the benefit of frequent commits, reviews, and research breaks; they might not prevent our ignorance, but they keep it under control.
This is true, at least, until you hit code that touches hardware directly, and then you still get to encounter lots of new problems because the hardware is doing things that break your code.
The times I've spent most time reading, are when I've transferred across industries, to capital markets trading systems, to web media, to web apps, to mobile apps, to big data, to VR/graphics apps. Each transfer involves an initial period of furious research, but even though the ratio of reading to coding stays high initially, I am generally able to start writing more than reading within the first few weeks.
I did spend more time reading the first years of programming. But I think you hit a watershed, where it becomes harder to find a treasure trove of ideas about software architecture, for example, in a new book, and instead it becomes more helpful to advance your art by experimenting, and developing, and analyzing your own output.
For what it's worth my main point was about the ratio of coding to research, and although the previous commenter disagreed, I think he also introduced a new point about the ratio of thinking to writing. I remember I used to spend more time staring at walls of code, or paper notes. Not anymore. If before my work was more chunky, now it is more consistent and smooth. That is a skill/discipline I developed. The article resonates with me because my commit frequency has increased since I have been able to improve my analytical process in this way.
But, the overall conclusion - small bites, many times - definitely fits both my experience and my intuition.
You either have to a) create a library/module or b) use a library/module. "A" is rare, and it's easier to see the impact of it - "you're the person that made that thing". "B" is the most common, and it usually involves many small bites. It's also harder to notice who's accomplishing what this way, since you have to point at the whole team to say "you made that thing".
Impact sounds like a decent way to see who's meaningfully contributing to a group project, although it definitely (sounds like) it has major blind spots. They could do with more detail as to how "churn" relates.