Hacker News Reader

Comment by mindvirus | Hacker News Reader

I think as a profession we've been bad at quantifying and communicating the value of clean infrastructure and good development practices to the business. Why should they pay you to make things "nicer"? One of your big customers is threatening to churn if you don't deliver this new feature! To them, it sounds like you are asking them for a ton of money for better feng shui - pseudoscience with unclear benefits - when they can directly see the benefit of delivering features to the customers.

Instead, I think we need to learn how to measure and quantify the value of a good codebase and good practices, and communicate that to the business. Google's error budgets are a step in the right direction here, but I think there's a lot more that can be done.

View on HN

> To them, it sounds like you are asking them for a ton of money for better feng shui

I've used the analogy of kitchen / food hygiene. It doesn't necessarily improve the taste of the food but it prevents cockroaches and death by food-poisoning.

Well put, thank you.

Not to mention being shut down by the health department, losing money and requiring new staff to learn non-standard ways of doing thing which takes longer AND causes an over-dependence on long term employees who are the only ones who 'know how to do that'.

Better fung shui? No, more like 'not cutting corners'.

Those are quantifiable costs though. If your restaurant is shut down or your reviews are bad, you can say how much it'll cost the owner. The owner can then make a more informed decision about how much to invest in cleaning.

I've been brainstorming this with colleagues at my job and tossed out the idea of including both an immediate development cost as well as a "long term cost multiplier" for every requested feature. So far, no one has the like the idea or phrase. I can't really blame them.

But there must be some quantity or term, with a nice memorable ring to it, that could get this across and at least let business people make decisions with full information at hand. Granted, any such numbers delivered in that guise would be wild guesses. But still, better than nothing.

Edit: However, I've since realized that the notion of "story points" and "velocity" already sort of provides a way to get this across if used carefully.

We need to quantify it though. Thinking about long term implications and talking about it is good, but that doesn't break us out of the bubble of communicating it to the business. After all, they don't know what a story point is (heck I don't really, and I've been doing this for a while).

As the article points out, management can be short sighted- they want to deliver on their roadmap this quarter or half (because that is what they are paid for).

I don't know what the answer is, I struggle with this. Testing has some promise here: if we can deliver (failing) tests early on that demonstrate progress toward the roadmap, and use those to show progress to the business and help them hedge risk, we can use tests as a forcing function for quality - testable code, especially for business processes, is usually good code.

I sometimes give two estimations in one: up-front cost of development and long term consequences. Could be something like,

"8 points + three support cases per week" for making a new feature without fully thinking it through.

"32 points + misconfigurations every other month" for including a complicated feature that can more easily be abused.

"We can go with our current best guess, and the effort of that is maybe 2 points + potentially 12 points later if it turns out we were wrong. Contrast this with 6 points to be certain of doing the right thing now."

In my experience, this is highly appreciated in environments that take you seriously, listen to you, and understand what "estimation" means,

Your competition gave an estimate of 2 points, did the work, got paid and moved on. Now you are supporting the feature...

If that was the right call, then great! If not, hopefully someone learned from their mistake. If nobody did, I'm probably not interested in that business anyway; it sounds like it is a recurring source of friction.

How can story points be used for that? To me that's only relevant in the short term. Can management be made to understand that velocity can be too high?

Edit: Expanding with my previous thoughts on this - as a software architect I've explained my role to management by saying it's like I work in an agile team, but the waterfall is in my head - someone needs to keep the architecture together and drive it towards green pastures where it keeps great flexibility for developing future anticipated features without giving up stability.

My thought was that story points can be used once they are turned into velocity by dividing by time. And you can say that adding this feature half baked now in short time, will inevitably drop our velocity by 10% in the future.

Like a huge afterburner that is a dead weight after 3 seconds...

Isn't that what tech debt is about? You write code that works right now, but you know it should be refactored because of long term costs. Keep track of it, put it on a wall, quantify it (days, story points, whatever) and talk to your manager about what tech debts need immediate resolving, what are the nice to haves. The worst thing you can do is to build up tech debt and be silent about it.

Tech debt needs to be handled, or else your software won't be maintainable in the long term and new features will cost much more and put the system in risk even more.

And there is a nice slide about it: https://www.slideshare.net/cairolali/resolving-technical-deb...

(In case you are wondering, the circle means, you're past any maintainability. Go to start, rewrite. If your client or manager wants to avoid it, he/she should plan for refactoring).

I hear you on tech debt, but in my conception, it is just one portion of "long term cost multiplier". Because even a perfectly coded and tested feature adds weight to a system and slows future development.

"Growth". That is one word that makes managers look like deers in headlights.

At least the first time you tell them. Maybe the issue is much bigger than just communication. Perhaps they also realize that they don't work for, or will get rewarded for, far future profits and benefits.

> However, I've since realized that the notion of "story points" and "velocity" already sort of provides a way to get this across if used carefully.

Not really.

What you're missing is called risk. risk with respect to missing the deadline and long term risk when doing a sloppy job.

Business people understand risk, yet software developers go through all this effort to talk about story points and velocity.

Your right, a large part of the problem is that there isn't a quantifiable way (outside of things like bugs/kloc, which are themselves suspicious) to identify a clean code base. I believe that is because much of it is personal preference, the same way some people find single family homes optimal while others like high rise condos. One person may think that adding some "clean python" code to an existing pile of bash scripts "cleans" it up while another may think that keeping the code base using a single language is cleaner while trying to minimize churn and bugs.

Bottom line, I frequently question much of what is being sold as developer productivity these days. It seems in my experience "this code is ugly/garbage/etc" is simply translation for "I don't understand much of this code". Engineers that truly understand the code base just go and fix it, usually with commits that are either small one or two line bug fixes with long commit messages describing how the system is failing and what is actually being fixed. Alternatively, they have "cleanup" patches that have extremely high removal/add line ratios which are replacing large chunks of difficult to understand code with small pieces that are easier to understand, and long commit messages detailing where the redundant functionality was. Very rarely are these changes to a different language/toolkit. But the evidence of skill in both these cases just about never include followup patches to "fix" new problems introduced by the previous round of fixes.

Yes, I’d say Chesterton’s Fence often looms large in these cases. While “build one to throw away” is often ideal it is contingent on deep understanding first, and even then there is significant risk of overlooking major flaws in a new untested design.

In practice I think the best approach is to primarily refactor incrementally in service of ongoing work; management doesn’t need to approve this, you just build it into your estimates as an IC. If technical debt is being taken on call it out early and loudly and secure cleanup budget in advance if possible.

Beyond that recognize that big rewrites are rarely justifiable. Even if a code base is complete shit, it’s most likely not worth it to rewrite. As an engineer this reality may put you in a lose-lose situation. Learn to separate your personal feelings and innate desire for order, and make an objective call on whether the code is workable and start looking for a new job if not.

But you can just rewrite that complete shit and keep your current job. And rewriting is very fun. I did it multiple times and don't understand those who're ready to switch job or work on a bad code. Isn't it better to embrace your desire for order than to fight yourself? There are some people who don't care, good for them.

> outside of things like bugs/kloc

Since #bugs is strongly correlated with loc I'm thinking this might turn out to be very low in signal -- so much so that you might instead just throw a dice to determine the "code quality" of each project, if using this metric is the alternative.

you may need to look at bugs/ksloc as more of an enterprise measure than one for an single project. when it gets down to it, this is the measure that will tell you if your organization is 'improving'; end of the line reducing your defect rate is one of the most valuable things to accomplish.

Assuming it's more noise than signal, increases or decreases to that measure don't tell me anything about actual improvement.

I'm all for reducing the defect rate, I'm just not convinced optimising bugs/kloc is meaningful.

Well anything done directly to influence bugs/kloc is bound to throw the whole thing off. I think the parent here was mostly interested in discovery rate, which also can be influenced by hiring/firing test and QA, or for that matter gaining/losing customers that use the product.

And everything being mostly equal, it could be a reflection of code maturity more than anything. The more mature product may have a higher bug/loc ratio only because more of the bugs have been found.

Which is why I mention there are metrics, but they all seem to have major problems.

Benefits of oil change are also invisible until it’s too late, yet no one has problems understanding it. Can we learn something from that field, maybe? I’m grasping at straws here.

Code is too reliable, in many cases. If you don't do your 100 and 500 hour PMs and change the fluids on a bulldozer, then it will pretty reliably break down or render itself into a smoking million dollar chunk of scrap iron, in spectacular and unavoidable fashion.

Software can limp along broken without anyone noticing for far longer.

One of the reasons that oil can more or less easily be validated is that there is a spec to the engine and the spec can be show to be invalidated when oil is not used.

Directly measured, the engine cylinder walls will grow wider and compression will be lost. Indirectly we can do an oil analysis and see the metals deposited.

One of the reasons we do not understand the cost of low internal quality code is that we do not know the "spec". Defect rates is one proxy measurement, but it's hard to know if it's just "bad programming" or "bad internal code quality" ...

> One of the reasons that oil can more or less easily be validated is that there is a spec to the engine and the spec can be show to be invalidated when oil is not used.

no, it's because the engine will seize and fail if you don't change the oil, and it's readily apparent to everyone when it happens.

But the benefits (and downsides) of doing or not doing an oil change can be measured. You can run tests on engines that skips or delays oil changes and see what the effects are and measure them in dollar amounts. Like cost to replace / repair the engine prematurely. Sure YMMV but you can get a rough estimate. What is the rough cost in dollar amount if I don’t do this refactor?

Definitely. Developing a unit of account is key to solving this problem. The problem was never that businesses didn't "get" technical debt, the problem is that they could only measure it by the volume of developers' whining.

If the business lacks sufficient capital reserves to continue operating through that customer churn then issues of clean infrastructure or good development practices become completely moot. The business will fail before the engineers have time to improve anything.

This is why I prefer to turn the problem on its head and ask the business how much time they want me to spend working on 'quality' (i.e. refactoring/tooling).

If it's on a spreadsheet that their boss can see saying that they asked me to spend 0% of my time on quality then I'm ok with it. If they can justify putting that zero down then presumably there IS a good reason.

If they're going to put me under implicit pressure to deliver as many features as possible and then cry havoc later when they get bugs then I'll lack sympathy.

Product managers are often resistant to the idea of tracking this number, however, since it provides a new angle through which they can receive flak from above.

What I meant by that example wasn't an existential threat (I agree if you're there it's already too late), but to contrast how the business people are thinking. As in, they're thinking "here's $1MM of ARR that's likely to go to our competitor away if we don't ship this feature" when the engineer is asking them to get time to clean up the code.

I'm not saying that clean code is a waste of time - it is incredibly valuable! I'm an engineer, and I value good architecture a lot. I'm saying that to get organizational support for it means that we have to get better at demonstrating that value to non-engineers, which will both hold us as engineers accountable to working on the highest impact things (is delaying features worth the potential short term ARR loss or slower customer acquisition? We should have a good answer for that) and help the business balance long and short term prospects.

Reply on news.ycombinator.com