Having metrics for something attracts your attention to it
utcc.utoronto.ca
utcc.utoronto.ca
If having a metric attracts attention, and (suppose that) attention eventually makes it a target, and that makes it a bad metric, then you can't have any good metrics for long.
Take some metrics, focus on one or two for a while, then focus on different metrics, come back to the old ones in a year.
There is plenty of precedent for this in athletics, and competitions of intellect. We just haven't connected the dots.
If you focus on bench reps the whole year it's going to fuck up your performance. If you focus on opening moves all year, it's going to fuck up your chess. That doesn't mean never focus on them. But we are stuck in the fallacy of the excluded middle, so we think we have to do something all the time or not at all, which is just the dumbest fucking thing this entire profession has ever come up with. Worse than PHP.
https://utcc.utoronto.ca/~cks/space/blog/tech/DangerousMetri...
Another use case I have found is providing too many metrics can lead you to focus on metrics which are not important. Panicky ops people will chase spikes on a graph, which don't represent a real problem. It can be tough to talk them down off the ledge.
> The first step is to measure whatever can be easily measured. This is ok as far as it goes.
> The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading.
> The third step is to presume that what can't be measured easily really isn't important. This is blindness.
> The fourth step is to say that what can't be measured really doesn't exist. This is suicide.
Cut only if you lack space/capacity and you have reasonable suspicion it absolutely won't be useful.
Sometimes you end up with a happy accident where you want to know something and the past measurements are useful, but pretty often when you start looking at the data, there's some reason it's not actually useful.
This problem seems quite common in software these days, and is why I've soured a lot on the use of metrics to aid development.
Just strap a device onto your body during the day and then it just tells you "you're doing great!" or "you're not doing well!" without delving into specifics. Of course the raw data can be analyzed by a medical professional who will make his own considerations without you meddling with the metrics
Some days I focused on my pace. Other days I focused on trying to estimate my pace without looking at my watch. Some days I looked at my heart rate. Others I just concentrated on my gate. What are my knees doing? What are my hips doing? What is my core doing?
But after a couple months, I spent about half my time just zoning out and doing walking meditation or enjoying the scenery.
The trick is not to fixate on any one thing. The opposite of fixation is not apathy, it's moderation.
In general, whenever you look somewhere new, you will find something you didn't expect. Science is just replete with this. In the funniest cases you get elderly scientists confidently declaring that some science is Done and All The Things Have Been Discovered... then someone puts up a new streetlight/metric and it turns out there were more things under heaven and earth than was dreamt of in their philosophy.
I haven't found a good word for this in general but I've been internally using the phrase "cognitively available" for this sort of thing. There are well-worn paths through the space of all thoughts we can think, and they are generally co-evolved with the world we live in and structured the way they are for reasons, but they are not always aligned with the details of where we are right now. There's a lot of variants on this theme, such as adding a new metric and freaking out as the article says, bikeshedding where a group would rather argue about something cognitively available and easy to argue about than the hard thing they really ought to be arguing about, the whole System 1 versus System 2 thing (google it if you don't know what that is), all sorts of manifestations of cognitive laziness resulting in a sort of psuedo-force attracting you to the cognitively available easy concepts instead of the correct ones you need to engage with.
Interestingly and topically, my experimenting with ChatGPT shows that it has a very similar phenomenon, and others have seen it too. For ChatGPT what is "cognitively available" is discussions that have been done to death on the internet. Ask it something super popular and it will definitely give you the right answer. Ask it a very closely related question but one that is different in some critical detail, and the "attraction" of the common question will overwhelm the model and it'll still give you the answer for what it finds cognitively available. (This suggests that mentally modelling ChatGPT as an extremely well trained System 1, but still ultimately a System 1, may be a productive line of thinking.)
Some of the things we do are good exercises, some are good practices. Confusing the two hurts, and avoiding exercises that make dubious practices also hurts, but takes a lot longer to show up.
If I go a year without doing TDD, I start to feel it negatively in my code quality. But if I do TDD every day for more than a few months, I also start to feel it negatively in my code quality. So when I get a story I decide whether I am going to do it or not, and I try to alternate between easy stories and difficult ones.
That is a stunningly brilliant distinction - simple yet deep.
I shall definitely be stealing this and passing it off as my own ;-)
Having worked on systems with intermittent bad nfs performance, the client side logging for NFS in Linux seems very poor since it’s handled in the kernel, client side reboots clear inode caches, and it’s not hard to wedge up clients with a recursive find (or worse recursive grep) commands when directories have tens of thousands of log files or hundreds of GB of files. The raw NFS rpc metrics are tricky to interpret but I am not at all surprised you see huge spikes. If you find a way to correlate those NFS metrics with what’s happening with the actual issuing commands or processes, you’re doing better than I and let me know!!