Hacker News Reader

Comment by austincheney | Hacker News Reader

Rewriting from scratch is usually, but not universally a bad idea. There are times in your life when you encounter that piece of legacy internal software that is simply garbage. The math is simple if it takes less time to rewrite than to refactor then rewriting is better, but also keep in mind that estimating software efforts is often inaccurate.

I have also noticed that in Agile teams refactoring is generally not rewarded and sometimes is even discouraged. Instead it is a voluntary effort taken at the developer's expense and risk, which is perhaps the greatest crime of Agile.

View on HN

> I have also noticed that in Agile teams refactoring is generally not rewarded and sometimes is even discouraged

As a long time Agilist, the spread of Fake Agile is a sad thing to see.

Constant refactoring is a core Agile practice! https://martinfowler.com/bliki/OpportunisticRefactoring.html

I always get nervous when “refactoring” shows up on the backlog and then gets deprioritized. In my mind it should never show up as a separate task and then it should never get ignored . It should just be a normal part of the workflow for professional development that gets done without much fuzz. Most long term technical debt can be reduced a lot by proactively refactoring small things that for some reason don’t work well.

When I worked closer to production the people at the workshops also cleaned and repaired their equipment every evening, crunch time or not. They didn’t just let their machines die long term because it may buy some time in the short term to skip maintenance.

This has been my experience. Carving out time for refactoring explicitly is only beneficial when your stakeholders, PM's, etc. understand its importance completely. Usually, at least one person up the chain says "it's already working and we have more important things to focus on. Put it on the backlog indefinitely."

Better to just incorporate refactoring into the estimate for any given coding task and do it without explicit permission. Unless of course you're in a culture that is willing to straight up miss a deadline in order to get refactoring finished.

“Better to just incorporate refactoring into the estimate for any given coding task and do it without explicit permission.”

It should be as implicit as attending planning meetings. For some reasons PMs usually have no problem forcing people to sit through meetings but they have a problem with spending the same amount of time refactoring .

In my teams we take tickets for meetings, which makes product owners very aware of their impact on delivery.

I like including refactoring in tickets, but if the refactor is consequent it's not going to work. It's not so often thankfully, I usually manage to convince the PO of the value of it, then try to not budge once it's decided

Evidently the answer is refactoring during meetings. /s

Sarcasm not needed. I've probably done this during a meeting. I definitely code if it's an all hands on deck meeting that gets hijacked by technical discussions that aren't relevant to my work.

I probably spend 1/3 of my average day refactoring.

Occasional there is a bigger refactoring project, but that's my normal day.

“Real agile” relies on a thorough suite of tests, a stable team that understands the domain and the codebase, and clear explanations of problems rather than a shopping list of solutions. Most teams don’t work in such an environment, so safe refactoring is impossible. Instead it’s just a whole series of shotgun surgery appointments.

Every "Agile" shop I've worked in was awful and has been doing it wrong according to Agile fans.

"That wasn't true Agile" is starting to sound about as credible as "That wasn't true Communism."

Best methodology: Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project.

That requires two things:

  1.  Hiring the right people.
  2.  Trusting them.

Most companies aren't willing to do this, so... we get a semi-unstructured form of micro-management hell which goes by the name "Agile." Regardless of whatever that term originally meant, this is what it means now.

> "That wasn't true Agile" is starting to sound about as credible as "That wasn't true Communism."

The difference is that many people have worked on highly functional "real agile" teams and can tell the tale.

> Best methodology: Self-organizing teams of experienced, expert engineers.

Minor quibble: While that is of course the best scenario, it can also work quite well with a few experts guiding the less experienced. Pair programming works wonders here.

If your team is all inexperienced newbs, I don't know what can save your project :)

> Regardless of whatever that term originally meant, this is what it means now.

Perhaps. I have no interest in debating word definitions though.

> The difference is that many people have worked on highly functional "real agile" teams and can tell the tale.

Sure, but is it really the "agility" or the people making the difference?

The highly functional teams I've worked on either ignored or satirized the elements of Agile development (e.g. the "story" format, story points, any sincere effort at estimation poker) to get out of those discussions as quickly as possible. This probably bears some similarity to how the original "Agile" developers dealt with the preceding methodologies.

Both these things are directly written in the Agile principles.

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”

“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

So based on your description those teams were Agile in the sense meant by the manifesto.

You know, that is a fair point!

The great Agile teams I was on had really good people on them. So, sure, they might have done great work in other environments.

Then again, this was the methodology they chose.

It also sounds like the teams you describe picked their own way of developing, which, in a Big Picture sense, is all the Agile Manifesto says: Let the team control their own work, and get out of the way.

Which, in a lawyerly and annoying, but still kinda true way, makes that team agile too :)

“Self-organizing teams of experienced, expert engineers. Let them develop whatever process works best for the individual team and project.”

That’s what “Agile” is when you read the Agile Manifesto.

It seems every reasonable methodology quickly gets perverted by leadership to a top down, dogmatic dictatorship caricature of itself. Happened to OOP, Agile, will probably happen to FP. You could argue that what Lenin and friends did in Russia was also not what Marx had in mind.

This is what agile proponents always say. Whatever someone is suggesting, that's agile. It appears to mean nothing. Here, you're saying that the agile manifesto literally says nothing beyond "let people make up whatever process they want", which boils down to nothing at all. If that's really the case then agile as a concept may as well not exist.

But we have story points, epics, scrums, retros etc. Agile is very much more than that.

I think people just confuse Agile the manifesto and various methodologies that claim to adhere to it. You can work in a way that meets the manifesto without any of those concepts that are tied to it through things like Scrum.

Most “no true agile” posts are really “no true scrum” because the issue is firmly in process and methodology rather than underlying principles.

In fact I’d say Scrum itself whilst designed to meet the principles doesn’t actually do so in practice.

You have to look at how Agile was started. Before it often projects were planned out in detail beforehand and then they tried to execute that plan without changes. As it turned out that often didn’t work. So the Agile Manifesto writers tried to explain that software is unpredictable and you have to make sure people can adapt to changes.

Scrum as a base methodology isn’t too bad either but as always management prefers precise plans and a lot of Agile consultants are happy to feed that. Saying “let people make up whatever process they want” is actually quite profound because it often goes complete against what management wants.

Even granting Marx wasn't particularly specific, the specific invention of Leninism, the Party as "intermediary" between the proletariat and History with a capital H, is conceived of absolutely nowhere in Marx's work, but the 'perversion' of Marxism to Leninism to Maoism is no different than the 'perversion' of Agile to Scrum to SAFE; there's no 'perversion' at all, because the idea of "lineage", while it works for and describes sequences of organic structures, is completely unfit for understanding ideas and how and why they change, to the point that its difficult to really use the word "origin" when it comes to ideas, particularly when each phase distinguishes itself from the last by what problems its trying to solve, the constraints on solutions, etc. We can only really talk about them being related insofar as there are structural identities in the concepts but that's about it.

If this is clear and understood, congratulations, you basically understand the gist of the French historian Michel Foucault's criticism of modernity (it supposes origins and progress when no such thing actually exists, so how do we talk about politics, history, ethics?), a criticism which would go on to inform all of contemporary Left politics, including contemporary progressivism and identity politics.

Then, to clarify, I almost never see story points assigned for refactoring existing code. Completion of story points is generally the metric of success in the absence of more valid metrics. That measure of success applies to the scrum master that moves the project forward and the developer assigned to perform the work.

> Although there are places for some scheduled refactoring efforts, I prefer to encourage refactoring as an opportunistic activity, done whenever and wherever code needs to cleaned up - by whoever.

That is how I personally view refactoring as well. Its like a necessary sanitation ritual to keep things tidy like doing the laundry or the dishes but far more rewarding knowing that the change is more than mere maintenance, an improvement that simplifies things. Refactoring shouldn't require a scheduled event, but its the quantity of scheduled events that is visible and reinforced by the system.

You're describing another sad disease. Or two, really.

1. Story points are only intended as a prognosis tool. If we produce 14p/week on average, we'll probably work through this 80p block in 5-7 weeks.

That's it.

When you start using it to evaluate teams or individuals, Goodhart's law sets in, and it all turns to garbage.

2. The reason you don't count refactoring or bug fixes as points is that they don't add user value. If you spend all your time fixing bugs you created earlier, you are not being productive to the customer. It's easy to get caught up in polishing your toys instead of delivering things the customer is paying for.

Though what's often unstated/forgotten ignored, is that refactoring and leaving the code in at least as good a shape as you found it, is an integral part of what calling a story "done" should mean.

https://en.wikipedia.org/wiki/Goodhart%27s_law

> The math is simple if it takes less time to rewrite than to refactor then rewriting is better

This is the common pitfall with rewriting. It does not account for the existing software being tested in production for years. There often are subtle behaviors/bugs, that users by now rely on.

This is why it's worth the effort to make behavior clean, coherent, and logical.

"Huh the behavior of this system is a little eccentric, but if I insert my code in just the right place, I can get it to do what I need!" This is how legacy systems get built.

Instead you have to ask, "How can I describe what this system does in one or two sentences?"

"How can I describe how it does it in a short paragraph?"

The first explanation should make it clear why you're building the new functionality into or on top of the system. The second explanation should make it clear how the system supports the new functionality and why it works. These two explanations form a team's shared understanding of the system. If you build a new system that conforms to this understanding, porting is straightforward, even if you don't replicate all the details.

Invariably, though, there are developers who are too lazy to ask how a system works, and instead crawl through the code sticking their new feature in likely-seeming places and leaving it in the first place where it seems to work, regardless of whether the reason it works is related to the shared understanding of the system. Because everybody else must keep this new feature working, there can be no understanding of the system separate from the system itself: there is just a mass of equally significant details. The intentional developers are reduced to the level of the random developers. There is no such thing as a rewrite of a system like this. All you can do is replicate it nut by nut, screw by screw, and then it still doesn't work because you didn't match the spacing of the screw threads.

The corollary to this is that if you build a complicated enough system, with an abundance of distinct places to plug in code, the random developers will always find a place to stick their code and will always be happy and "productive." This is the way to achieve architectural immortality inside a company: create something so complex that it can't be understood but solves every problem if you look hard enough.

In some cases, it's possible to rewrite incrementally via refactoring. (Whether you can do this depends on the architecture of the system and whether the starting and target languages have decent interoperability.) In other words, rewrite one small piece of a system at a time while maintaining the entire unit and integration test suite. While this doesn't completely obviate the risks you point out, being able to rely on existing tests can make the effort far more plausible and less risky.

Joel's examples of hairy functions. "Why does this need to be 50 lines and makes half a dozen case exceptions? Just strip out that noise, god who wrote this?"

Now legacy systems don't cooperate, well I guess they can just rewrite their APIs too because "software entropy".

So, unmaintainable undocumented untestable legacy code? Yes please, rewrite it, break it, and when its real intended behavior is understood, write tests and document it so the next guy doesn't have to wonder about that crazy function..

Please stay off my systems. If you deploy broken software that runs your business, no one has to wonder about any of the functions any more.

If you keep doing that, there won’t be a next guy. The company will be bankrupt.

That's why commercial software is doomed to be crap.

Why are you “rewriting” instead of refactoring? Even if the existing code is “garbage”, if you need to add new functionality, you can usually use the “strangler pattern”. Put a nice facade over the old code to provide an interface and write on top of it.

If I can do it by putting a COM wrapper on top of 20 year old PowerBuilder code and then interact with it with C#, I can’t imagine a case where this can’t be done.

I was once partially responsible for maintaining an old .NET remoting system that did absolutely nothing to isolate its transactional boundaries.

The services were baked into the model which made it easy to write in the first place but very hard to refactor. The code would load a "hollow" entity and then as the entity was interacted with: automagically over the wire the data would be lazy loaded. If you applied an SQL trace you might see over 300 separate SQL commands executed just to load a page that visually represented the entity.

This resulted in some dire performance outcomes, a simple for loop over a given collection on the entity could result in N round trips and sometimes even N*N depending on the data modelling in the back end for given types of data within a collection or association.

A primary issue of this system is that lots of client code had been written that didn't express its intent. There were about eight relatively complex clients of this software and nowhere in that code base did the client ask for the data it wanted upfront in a way that would allow someone to refactor it without rewriting every single one of those eight clients.

Now yes, you could refactor this "architecture-less" sub-system by replacing it with another "architecture-less" sub-system that was "better", but sadly you're left with the same problem: the clients will never tell you what data they need for a use-case they will just assume it will appear when its accessed. To add in those "data demands" you're performing huge shotgun changes across the entire code-base which are always too expensive to justify.

One particularly egregious example was a WinForm loading an entity and then stuffing it into the .Tag of a control where a handler would cast it back out later and access it again in a lazy-loading kind of way.

Your only hope in that case is some sort of static code analysis to attempt to reverse engineer the client's data demands but that's probably approaching as expensive as any of the changes.

Microservice architectures, if you have the organizational scale to support them, are intended to produce services that can be rewritten and replaced.

I've actually done this on purpose, by getting a v0 API into production quickly (1 week), then getting a v1 with all the bells and whistles out with sufficient time. It completely replaces the the v0 microservice which is a separate deployment.

The first version provided a specific implementation of the functionality for a single, small user population. The full version scales to millions of users, globally. It is still small enough that we could rewrite it, if we had to. The infrastructure is actually harder to produce (as code) than the API functions themselves.

CD pipelines, extensive unit tests, automated smoke tests, automatically configured health monitoring... all of these things make it possible to replace a service with a different implementation that conforms.

It's a hard discipline to get to, but once you're there, it is often more economical to rewrite to move on. I guess that's not the case for the usual kinds of projects. We had to spend a year as an organization to get to the point where this was true. It's usually not worth the cost for small teams to do this.

That’s all true in theory, but how many organizations have badly written code that needs to be refactored and good dev ops practices and/or unit test coverage?

And that still goes back to my point about using the strangler pattern. You put a facade on top of the “legacy code”, write integration tests that capture existing functionality and you slowly start refactoring the code or replace entire subsystems as necessary.

As far as v0 versus v1. I did just the opposite. My microservice was written in Node/Express. It was our company’s first Node app, we were a Windows/.Net/IIS shop.

I could easily get buy in for Node because everyone was expected to know JS. But I would never have gotten buy in to set up a Linux server and I didn’t want to run Node on Windows.

I deployed the API on AWS lambda even though I knew it wouldn’t be good enough for phase 2 - 6MB request/response and the “warmup time”. I also didn’t know anything about Docker it Fargate (AWS Serverless Docker).

We got the first version in, I spent some time getting up to speed on Docker, Fargate, and a deployment strategy and rearchitected the entire back end hosting and deployment to use Docker and Fargate.

I always wonder how micro services will do once a refactor doesn’t impact just one service but several services and teams. This seems to be a recipe for a lot of political machinations.

Part of the expense of microservices is that they be individually replaceable. Every microservice has dependencies to the interface of other services, and only the interface, which is typically RESTful. Refactoring in one code-base (read: one microservice), would never effect another code-base.

The unit of reuse in this architecture tends to be the entire API, or libraries published as open source. If there is generic code that is suitable for a library, it is also suitable for contribution. If there is a core business function that should be reused, it is a microservice with an API and gets shared and consumed that way.

Our hard rule for refactoring or enhancing services is if it is a breaking change: use another name (Rich Hickey's recommendation). If you provide more or accept less, fine. If you provide less, or require more, use a new route (like a /v2...).

Therefore, refactoring in your code base, which makes your life easier, is transparent to everyone else who depends on your code. We have many small teams, so each team is providing 5 to 10 microservices, and refactoring within each independent repo is simple and low impact.

There's no magic here, we simply paid the upfront cost of adopting an architecture that can be fielded by a very large dev organization, with a concerted effort in high quality DevOps practices. This move was also a means to scale the organization without dragging our efforts to a screeching halt with maintenance burden.

If you have to change the API signature and not just add more information, you have to version it and have a decent deprecation schedule.

It's not as easy to just refactor when it comes to UI oftentimes because you are often writing completely new UX.

True,

It was probably bad wording on his part when he said “rewriting from C# to Angular”. If he meant, “we created an API in C# and decoupled the MVC rendering from the business logic and rewrote the front end in Angular”, that’s different. Since he didn’t mention rewriting the back end, I can’t assume they rewrote the backend.

I agree. I recently worked on a price where 1 java class had 14 methods named construct<subtype>fields which all built a html+js+css mess using a string builder. I was asked to add support for 3 more subtypes. And this was the god class for the UI. We rewrote the whole UI in 3 months. There was no way that could be refactored.

A rewrite that small is a refactor.

That just sounds like garbage code to begin with.

Reply on news.ycombinator.com