Metrics-Driven Development
sookocheff.com
sookocheff.com
This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem.
In the included example, latency is up on a given method. Lets say it turns out that particular process calls three different "manager" classes, and has another inline sql query. So, you do the right thing and spend a bunch of time cleaning it up and getting things looking more right (with no help from MDD, since there's nothing about development practices in this outline.)
Turns out, it wasn't the code after all. Someone had designed autocomplete to work off of a sql table, and someone else disabled the job that re-indexed the search column.
* Long story short, TDD and BDD give us development tools
* MDD appears to give us... help making minor decisions on potential focus areas? Maybe ammunition to tell our PMs what we should work on?
Anyone else see what the big draw for this would be?
>This looks like the epitome of reactionary development. Plus it doesn't lend you any real insight into the "why" of a given problem.
How do you know those infrastructure projects are the right projects to do? Will those bring better business value than something else (bug fixes, etc.)? Noting that developer happiness is a measure of value, maybe they are.
The only way to really know is to measure it. Put telemetry in there that tells you how your current infrastructure is affecting your business.
Does the poor infrastructure lead to more bugs, costing the business more in developer time that could be spent on new feature development? Does the poor infrastructure drive down customer retention or new customer acquisitions?
I think the author's point is that if you don't know what people actually do with the code you write, you can't prove that making changes will provide business value.
Being proactive instead of reactive is very difficult if your product managers don't know what your customers find useful. They can't know what your customers actually use your product for without metrics.
You simply don't, and are circling "instrumentation" and "measurements" in hope they tell you anything useful. But all measurement in the world won't help you know the value of something that does not exist yet.
Instead, you take a risk. Every time you are deciding what to create, you are taking a risk. People that keep claiming for those reactive methods just don't like big risks. No problem with that, but they also won't see any big rewards.
>You simply don't, and are circling "instrumentation" and "measurements" in hope
> they tell you anything useful. But all measurement in the world won't help you
> know the value of something that does not exist yet.
Dev Manager enters the VP's office, "Hi Bob, dev team X cost roughly 1 million USD a year, they'd like to work on an infrastructure project, current estimate is 4 months but you know how it goes, it'll be six by the time it's done. ",
Bob replies, "OK, so 500k of cost, what business value will it deliver?"
"I dunno. Well they actually say that you just take a risk, cos every time you make a decision you're taking a risk - and without big risks there are no big rewards".
"Yeah, I'm not taking a big risk with my familys future by being fired for not having any sort of defensible thesis as to why we decided to work on this project versus any of the others that other departments are screaming for. So thanks but no thanks."
"Oh no, but you don't understand, they're extra special. Team X are superstars they can't possibly measure their contribution like other business functions are expected to. Cos you know it's tech - it's like magic and if you don't get it you're just not smart enough".
"Yeah, they might be smart, but they're not smart at making arguments that will help me fund this project. Come back when you've got a good argument that I can defend to the CEO with a straight face"
... and that's the problem, big risks sound great when you're not on the hook if it doesn't come good. Ofc if you live in Google (or similar outlier) then you probably don't care because money is showering in from the heavens. But most businesses are in reactive mode because they're trying to steer between the various rocks of problems they face!
I'm not arguing against measurement or thoughtful decision making. I'm saying it's not a good way to engineer a product.
What would you do if Ford practiced this when designing a new car?
"Hrm.. metrics show that gas vapors leaking into the cabin lowers sales... I guess in the next version we shouldn't put the gas tank inside the cabin. The previous metrics said it would save money by reducing overall material costs!"
But I agree with you that most businesses (I'v worked with) are in reactive mode, including infrastructure!
So MDD is an office politics tool?
TDD and BDD are development tools that help you write better more maintainable code.
If all MDD is saying "put in metrics to help you win arguments with your PM", then I already am a MDD master. I'm not saying measuring and testing aren't important. I'm saying it's not a design paradigm. I don't know of an engineer who doesn't run tests before diving in to solve a problem on an existing system.
If you start from scratch, how does MDD help you build a restful API? What about an authentication microservice? Adding a new feature - what metrics can we measure? TDD and BDD BOTH help you with the process of engineering stronger solutions.
Saying "measure things and use analytics to make decisions" doesn't really qualify as a paradigm.
As described here, it seems largely a way to monitor for and address operational issues before they become "hair on fire" crises, though with the right metrics it could also be a way of focusing development strategic priorities around the business value more effectively. OTOH, what both this article and the linked, more-detailed overview of MDD leave out -- and which is critical to either of these uses -- is any deep discussion on criteria for identifying the right metrics aside from "devs know best" (I also think that the focus on developers identifying metrics is probably wrong unless the definition of "developers" is at least broad enough that it includes DevOps.)
There's some beginnings of ideas that may have utility here, but there's not really a clear coherent well-developed methodology, the way the name "Metrics Driven Development" suggests.
https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-...
This probably works for systems where it's easy to have meaningful metrics. There are plenty of scenarios where metrics are not that easy unless you come up with some bullshit metrics.
And... You get what you measure for.
Metrics always skew operations towards what is being measured, so don't leave out anything important, or it will be optimized away in favor of things that are measurable.
If something important is difficult to measure, it is a recipe for creating an organization that is very efficient at doing the wrong thing. (Queue up: Steven Covey story about the difference between being efficient and being effective.)
Having been subjected to metric-of-the-month management, I've seen bad metrics drive bad engineering practices. Everybody who invents a new metric thinks it's fair and comprehensive. It's not. If you tighten the screws and tell people to lower metric X or else, they're going to do whatever they have to, from short-term fixes that cause trouble in the long run, to gaming the numbers.
Has the "Build, Measure, Learn" development cycle fallen out of favour? Or perhaps, it just applies to a subset of business models.
Granted, that may just be my bias towards some manner of top-down vision/guidance speaking. It may be that the global maximum I was imagining never actually existed.
I do like the emphasis on single point of truth when it comes to metrics. I think that idea is integral to any software development process that wants to have a bus factor > 1
"If the system is not easy to monitor, it is not well designed"
http://benjiweber.co.uk/blog/2015/03/02/monitoring-check-sme...