Hacker News Reader

Comment by recursivedoubts | Hacker News Reader

i worked at a company a while back that decided it was time to refactor due to application complexity, which was driving developer productivity down significantly

a team of elite coders and a senior architect was put in charge of the project, and they decided to implement IoC & OSGi throughout the code base and split the repos up into cleaner modules

a year and a half later, they were done. glowing company wide emails were sent, it was a new era

things got significantly worse: the new system was very difficult to understand, testing cross-module issues was impossible to coordinate, OSGi added almost no value at a heavy cost across the code base, etc. developer productivity, already low, fell even further

the senior architect left shortly after the project was completed

careful with that refactor, eugene

View on HN

The rewrite should be done by the people that will be maintaining it, not "elite coders" that aren't going to touch it ever again afterwards

Almost every split between maintainers and creators will kill your productivity stone dead a week after delivery.

Not only have you guaranteed total loss of knowledge, someone who had never maintained anything has no chance of writing code that survives contact with production.

You'll get all sorts of shiny frameworks and nice architectural dreams, but no logging, no monitoring, no validation of the actual user requirements, no nothing. You spend the next 5 years rewriting from scratch if you're lucky.

I've noticed organizations with lots of tech debt promote exactly the people that created the problem in the first place.

Giving that group a big budget and autonomy to fix the issue is like paying tobacco company to cure your lung cancer.

The people who create a lot of tech debt (and other problems) are often highly visible and appear to Get Things Done. In fact, they may actually get things done, but in a poor fashion (in this context).

Repenning & Sterman, Nobody Ever Gets Credit for Fixing Problems that Never Happened [PDF] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... (pretty sure this is the one I want)

Being proactive and doing the right thing (maybe, potential higher costs earlier than they can be supported for instance) is a low-visibility activity. Even if it's your job to do the cleanup work, the one who wrote it first is the one most likely to get the credit no matter the quality. Because they got something out there for customers, not you.

A funny comic describing this exact issue: https://twitter.com/shreyas/status/1391594520207233024.

That’s right if someone shipped a buggy unreadable codebase the first time what makes anyone think they are actually capable of doing the opposite second time around?

Companies exploit their workers, and sometimes workers exploit their companies by selling them a dream and leveling up to a new gig. I would be wary of hiring someone who came off a project like that who bailed from a position just as things were delivered.

I’ve seen some of that as well. After a very large refactoring is done it is initially claimed a success by the lead but to only slowly dawn on the rest of the team that the codebase is not better, just more complicated without any significant gains. Mess is still there, it’s just a unfamiliar one. Questions to the lead of the refactoring project cannot be answered as they are long gone some other place.

Of course that could be the case, but in my experience , the inverse is way more common. A team is tasked with rearchitecting some part of a system, and by the time they’re done, everyone is burnt-out and one-after-another the engineers leave the company.

The goal was never to “level up”, it’s just that these large deliverables take a lot out of you.

Leaving during the middle of the project would be worse.

This is an attrition issue. The company lost an employee, fair game.

It actually seems like a good sign to me that they completed their current project before jumping ship.

I'll argue that things like this have nothing to do w/ the merits of refactoring in general and are usually due to poor project management.

A small team orchestrating a large scale refactor in isolation for over a year seems appealing to project managers because it allows that team to focus without interrupting the pace of feature development, but it makes it exceedingly likely that the output of that effort will fail for these reasons:

1. Evolving requirements. A refactor is a design that is focused on solving challenges _at that time_. Things happen over a year. A product grows, its supported feature set requires rethinking the nature of the domain, the pressures of scale force new considerations, teams get more rigorous about the kind of automated testing they practice, etc. A team that doesn't have a feedback loop into is likely to build something that may have worked for the system of a year and half ago, but no longer makes sense.

2. Lack of buy-in and knowledge sharing. In isolation, most software engineers tend to solve their own problems. Senior engineers often make assumptions about the knowledge of the rest of the team and about the coherency of their abstractions when they don't have to teach it to anyone or battle test it in any meaningful way. This is why collaborative processes like code review and pairing are often a huge win, but sharing these efforts needs to go farther and a team should be pushed to try incremental adoption and feedback until there's more confidence about a refactor. Large scale refactors should be chunked and interspersed w/ a workload that attempts to validate the success of the design (i.e. working alongside teams doing feature development to make use of the refactored design). And perhaps most importantly: the team working on this needs to have clarity on what it means to be successful. How do they know that their proposed design is meeting the needs of the team at large? Infrastructure, dev ex, or any team that is orchestrating something like this needs to think about their task like a product team, and "what does success look like" needs to be a question from the onset.

problem w/ that is that "is refactoring a good idea" becomes unfalsifiable: you can always blame something else

this is sort of like w/ the agile folks: when an agile implementation fails, "oh, you didn't do agile right"

obviously, better code is always desirable, but the temptation to rewrite is very strong in developers (I feel it all the time) and experience suggests it often leads first to a worse situation, and then a worser one

Like everything, you need to have clarity on what you're trying to achieve and when to pull the ripcord. I'm saying you need to frame the task of a large scale refactor so that the planned implementation is falsifiable.

A rewrite is usually a quagmire if you don't do these things, but even the need for a rewrite is usually avoidable. If a team is making frequent observations about pain points and places value on fixing them quickly, they can often be avoided. But there are discontinuities: product concepts and features that dramatically change the constraints of a system late in the game, unanticipated growth requiring you to reconsider architectural decisions, etc. There's a long tail of events that you could be proactive about, but would likely create over-engineered solutions that place more limiting constraints on the design if you tried to be. They still occur though. It's very important to sufficiently interrogate the question of if a rewrite is necessary to solve these challenges, but you also need to interrogate the cost of band-aid solutions w/ limited applicability.

I don't think there's a silver bullet here, but most of the failed attempts at doing this that I've seen have been a result of teams that didn't sufficiently consider what the outcomes they were trying to provide were and how to test their candidate solution more rapidly.

Would be interesting to hear more about this — why did they think the new way would improve the situation? Why was the new solution harder to work with? Why was it hard to understand?

Wrong refactoring? Added new unnecessary complexity? I’ve seen refactoring gone wrong, not specifically OSGi, as I have no idea what it really is.

Reply on news.ycombinator.com