Comment by thatwasunusual | Hacker News Reader

Comment by thatwasunusual | Hacker News Reader

Parent Full thread

I had a similar experience in the Norwegian (hello, neighbour!) public sector, and it was _also_ a rewrite from a partly COBOL-based solution. The difference is that we succeeded, which is - if you're Norwegian - almost unheard of. :)

The whole thing was rebuilt using new tech; Java for the most part, cloud-ish hosting (our "own" bare metal servers and OpenShift via a Norwegian company), and this stack was never a problem. In fact, not much was a problem, because:

The old system was _riddled_ with unit tests that could easily be replicated in any language. All of the bugs and/or minor issues over the last 30 years had been documented and had unit tests covering them. This was what saved the rewrite. Our rewrite could rely a lot on TDD based on those tests, which made it possible for us to trap problems early on.

Without those tests - written by programmers who probably are long dead - the rewrite would never have succeeded, or at least not in the time frame given, and at least not on budget.

Since then, whenever our consulting company is involved in anything that has to do with existing software, we ask the client about how their test coverage is;

* None, but you will give us $1 million per hour? Nah. Doesn't matter. Find someone else.

* Lots, and you'll pay us $1,000 per hour? Tell us more.

So: create (unit) tests for _everything_. Fixed a bug? Create a test for it. Changed some behaviour? Create a test for it. Removed a feature? Create a test for it.

EDIT: Final stack looked like this:

* Self-ish-hosted servers (non-Norwegian cloud providers was not an alternative because of privacy issues).

* Java as main programming language on the backend, but many services was written in C#.

* Frontend didn't have to be fancy, so some simple JS-stuff there; think auto-complete, no fancy DOM-handling or anything like that. Application as a whole was very server-centric.

From what I've heard, they are now trying to get away from Java, and move more and more over to C#. I just hope they keep the unit tests. :)

This is such a good approach. A former colleague of mine built an entire scaffolding of integration tests surrounding a test-free system that needed to have a large fraction rewritten. Mocking clients, collaborators and everything.

The tests were only ever used once for that rewrite (the customer terminated the contract shortly after the rewrite was complete, and the entire codebase was thrown into the rubbish bin) but I still think it was worth it. The tests didn't even run that many times because most of them passed at the first attempt but I suspect the effort of enumerating cases had a strong impact on the successful design of the rewrite.

Spend the first four hours sharpening the axe, and so on.

The first thing I do with any bug report is check for or write a test reproducing it. I’m entirely bewildered when programmers don’t do this. It saves time in so many ways, from explaining PRs to refactors or getting future colleagues up to speed.

> Spend the first four hours sharpening the axe,

Great analogy. I'll remember that one.

A legacy system with tests? Now that's something you don't hear everyday.

Is it really legacy, if it has tests?

Is it ok to ask which company provided hosting?

It's OK to ask, but I hope it's also OK that I won't answer. :) I can't divulge this kind of information on a project that I worked on. NDA etc. etc.

Also, after some DNS-digging just now, it seems like they have changed to a different (Norwegian) provider.

> The old system was _riddled_ with unit tests that could easily be replicated in any language. All of the bugs and/or minor issues over the last 30 years had been documented and had unit tests covering them. This was what saved the rewrite. Our rewrite could rely a lot on TDD based on those tests, which made it possible for us to trap problems early on.

This is the crux of it. The hard part of a rewrite or major refactor is not building out the new stack from scratch using all the latest tech. The hard part is getting the initial system in a place where it can be rewritten. This includes -

- Adding robust testing, both unit and integration/system/functional.

- Working, functional CI/CD pipelines that execute these tests. I have encountered so many legacy projects without CI pipelines, some of which even had tests but were just expected to fail. Not marked in the code as expected to fail, mind you, just expected to fail by developers, who shrugged and only ran tests they thought were relevant to their code change.

- Code cleanup, particularly if you're doing a slow migration by functionality (which you should be). If you're making a "user microservice," you need to structure the code such that all user-related functionality is isolated and written in a way that works as a microservice, before you write a single line of the microservice itself.

- Infrastructure to do parallel implementation[0], including comparison and reporting of differences in results.

The main problem with all of this is that it's boring. Rewrites are almost always, of course, advocated by developers who have to deal with the code on a regular basis. If you tell developers that they can do a rewrite, but first they have to spend 6 months writing an integration testing framework and fix all of the broken unit tests, most won't go for it. They want to jump directly into using Kubernetes on the cloud with Erlang.

The problem compounds when you look at the wider industry, because honestly, it's often a great career move for the developer who starts up a rewrite with Kubernetes and Erlang. They can stick around for 6-18 months, put "Rewrote legacy system using $HYPEDTECH" on their resume, and start shopping around. Doesn't matter if the rewrite ends up being a dumpster fire, which it normally is. That's a much better story for your career, unfortunately, than talking about how you deeply understood the domain and the quirks of the classic ASP code powering your current company's most critical software.

[0] http://sevangelatos.com/john-carmack-on-parallel-implementat...

Reply on news.ycombinator.com