So if strangling is out and big-bang rewrites are out...
What do we do?
What do we do?
Certainly, there are times where that's not reasonable. Maybe the core codebase is on a proprietary technology. Maybe it's built around an EoL framework. Maybe it's built on on a FOTM language that you can't hire for at all anymore.
In those cases, which are more rare than developers like to admit, I think a piecemeal migration makes sense. In my experience, it's better to do it by altering the consumers. Whether it's a frontend that can point to another endpoint, or an API gateway that can switch out the service it's pointed to, or making a shim layer (proxy), that can serve that purpose.
Having a service that is both the proxy and the new application is a poor separation of concerns, very difficult to reason about, and makes it too easy to intermingle the logic between the old and the new. In my experience.
For example, refactoring "to provide abstraction so future work is easier" is 90% of the time an error.
The end result was me losing my "mojo" after a year of producing effectively nothing. I'm back together now but my god it's a terrifying feeling when you've been programming for over ten years and one day you can't make the code flow out of your hands anymore.
I must have missed that in the first read. Yes, do not do this.
If anything survives the hype cycle for microservices, I hope it’s this: if your company isn’t surviving more than a couple of technology cycles, especially when they seem to be getting faster, then what are you doing this for?
Stop trying to replace one system with another one. Yours will be old and busted someday too. And certainly don’t let one system be the gateway to the other. Put them both behind a thin layer that handles only a couple of concerns (say, auth, making sure all requests have a correlation ID, perf statistics, maybe pick 2).
What we are both suggesting, I have still always called the strangler pattern. It just has three actors instead of two.
That makes sense, in practice I've seen multiple situations where it was one "monolith" directly in front of another, and rather than a strangler, it was a tumor that ended up with things intertwined. Treating it like a service oriented architecture is a good way to think about it.
You still end up in situations where logic needs to be shared. Do you duplicate Auth? What happens when a new user signs up, do you sync users over? Do they read from the same database(s)? I'll also agree with the over hyped ness of microservices, but I think breaking out services and "rewriting" them to me is the way to achieve the overall goal. Even in the strangler pattern, you'll end up needing to share functionality, and either you have to monolith services calling into each other, or a service oriented architecture.
Authorization gets duplicated, but that’s going to happen at some point anyway (and how many times does it get duplicated in a microservice architecture?)
I’ve seen the tumor thing too. A few of the most egregious services (memory hungry, least coherent to the whole, etc) get split off to be independent, but lots of things do not. And sometimes the old team sheds people due to it not being the cool thing anymore and then you’re really fucked.
[edit to add] I have developed some very serious resistance to the 'low hanging fruit' model of development. I think it is the direct cause of the Lava Flow Antipattern.
If the point is to completely remove a problem from the system, starting with the most popular one and working toward the more boring ones means that at some point you will lose the argument in a prioritization meeting, the lava will cool and you'll be stuck with both. I would expect a higher likelihood of success if you start at one 'end' of the system and guarantee that high priority examples are evenly distributed throughout the effort.
At some point you have to ask how the company let things get this bad, and what they've done concretely to avoid it happening again. And whether you really want to participate in the heroic levels of effort it's going to take to keep the patient alive in 4 years.
Louis C.K has a joke about how his ankle was bothering him and the doctor told him, "Well, it does that now". That's it? "It does that now?!" Monologue monologue. The first time I laughed along with his joke. On subsequent viewings I said, "Hold on..."
Louis is a seriously sedentary guy. He's been letting things go even more than I have, which is saying something. But I do exercise. If I get to pick the feat of strength, I could make the leaderboard. When I bunged up my hip I got PT, and I went, and I could do the exercises. I didn't do all of them though, so it still bugs me from time to time (also on the last day the PT did something to my good knee, which is still bothering me a year later). If I were in proper athlete shape, they might offer more, because my prognosis for recovery from something more aggressive would be high.
Nobody is going to look at Louis, or listen to Louis, and offer that. Louis is not going to do any of that shit. He's just going to crack jokes about it, or about you... as is amply evidenced by that set. So yeah, your ankle does that now, sport. (BTW, my ankle is fucked up too, and I do the exercises he pantomimed. They do actually help quite a bit, which he would know if he had tried them even twice)
And yet we do it at work all the time. And I am seriously beginning to wonder what would happen if we just triaged these situations and walked away from some of them. If we just let the saner competitor win.
I read it like that at first. Think about it, it's also very true.
We should probably get that checked out.
Because what I've also noticed is true with respect to both of those phrases is...
It. Never. Actually. Does.
There are only so many people you get to teach in your lifetime, and only so many projects. If you only take the easy ones you never grow and you will reach fewer people. But if you take every hard luck case you also won't get far either.
A working relationship can be broken the way a married couple can be wrong for each other. Sometimes divorce is the correct course of action.
It's not enough to be right, you have to be productive too. And sometimes you only get both if you move toward people who are closer to agreeing with you.
Nothing is perfect anywhere, sure, but there are degrees.
You get the spaghetti when you take your eye off the ball. When you start making excuses for why we are going to do dumb things just “for now”. And by the time you notice the three year old comments that say “this is temporary”, well, your processes are set. This is the culture, and if you make a stink, people are thinking more about “why now” than “why not”.
The only way I know to get out of that is starting the conversation early about how we can do the things we always do faster and more accurately, and how we can do things we’ve never tried before. But that tree takes a while to bear fruit. And it requires coworkers who are willing to try something new, because for sure as shit nobody has shown them how to do this before.
Devs: Oh yeah... It does that now.
LMAO.
In the words of Monty Pyhon it was a dead parrot, and I came in to say "It's just resting... It's pining for the Fjords!"
I'm almost a month in on a complete re-architect from scratch. Most customers are overjoyed that the business will be back soon, but I'm afraid their good will is going to be short lived if things aren't running soon.
I am sitting on a codebase whose oldest line of code is about 20+ years old and has evolved successfully in that time such that product it was 20 years ago and todays product are unrecognizable from one another. Its database schema is even older, encoding decisions made 30 years ago. Reasonable decisions at the time, but no longer reasonable.
I am working on a major rewrite which will take a year. The nature of the changes required mean I cannot break it up into pieces and do it bit by bit as I have been for last couple of decades with major enhancements. A functioning product is all or nothing. As someone who is anti-rewrite, pro-evolve and accustomed to working on old codebases, and being my own business so its my own money on the line, the decision to embark on a 12 month rewrite is not taken lightly.
It is a refactor, but a refactor that is going to take about a year to complete, where there will be nothing to show for it until it is finished. From old codebase, probably about 20% to 30% of it will be ripped out and replaced.
Its not my first major project in terms of keeping this codebase capable. Previously written compilers/runtimes and IDE modules to keep it going : to gain control of codebase which was written in a commercial/proprietary dev environment into something I have full control over. That effort took a year.
The difference in this case is I am deliberately, intentionally throwing code away, alot of it, for the first time and contrary to my instincts. Still a post mortem might be interesting. My successful efforts to code my way out of a proprietary dev environment was an interesting and risky project, discussed at length with relevant dev communities at the time, but probably worth writing up one day.
With regard to your own project, I find it somewhat hard to believe that you can rewrite a 20 year old code base in just one year's time. Assuming an output of 50 LOC a day - and 250 work days a year - which means the system you are replacing is under 15k LOC.
1. Strangler pattern - which allows you to continually advance functionality, but will take longer and requires delicate care.
2. Complete rewrite - the only successful way to do this is to code freeze your legacy app. It's risky because it means keeping the product features stale for awhile, but less cumbersome to advance once it's done.
Either way is risky. Choose your poison.
When you accept that the old code is expensive but not actually impossible to modify, you can do an incremental rewrite without Strangler-style proxying, and not even needing to prioritize replacement of the user-facing components, and only or preferentially rewriting as necessary to make user-visible feature improvements or bug fixes, unlike Strangler Pattern’s preference for no-visible-effect initial replacements. (I call this “standard Ship of Theseus replacement”.)
This typically is even better for continuing to deploy new features than Strangler, but does leave you with a system without a single clean boundary layer between new-style components (the replacement/proxy system in Strangler)) and legacy, instead you have a system with mixed islands of legacy and new components (possibly multiple styles of legacy, if a third set of standards is adopted before replacement completes.) So, again, it's has its own risks/costs, but it's a third option.
You mean the one for the legacy? (Which I wouldn't call the framework of choice, since it's inherited, the new one is chosen...)
Sure it requires that the legacy code is, at least internally, supported or at least supportable, and sometimes that's not the case. Though on larger production systems, letting them get to a completely unsupported state is rare, and those are the systems where the choice between big-banf rewrite, strangler, and a more free incremental replacement are most consequential, IMO. If nothing other than a big-banf rewrite is possible, there's not a choice, much less a consequential one.
Example - you wrote your app in a custom PHP using PHP 5.6 and you're doing a rewrite in PHP v7 in Laravel.
That's often the case, but the app itself is usually actively (if lightly, sometimes, because the cost of change is high and there is a lack of skilled maintainance personnel) even if the underlying platform is outdated and possibly even out of support (and in some cases long abandoned by the vendor.)
> That's my point, you can't incrementally rewrite that because you'd be rebuilding on a burning platform.
You absolutely can maintain software on an outdated and, often, unsupported platform (the latter can sometimes be a licensing problem, as you may not continue to have the legal right to use it.)
Huh? I mean yes you can technically continue to use an outdated framework, but what do you do when there is CVE identified and someone exploits it? Do you just sit on your hands and wait until you can upgrade the whole framework months later?
There are explicit risks associated with continued use of an outdated and unsupported framework, so why anyone would continue to build on a burning platform is beyond me...
Technically, you can do a big bang rewrite, but they fail at a very high rate on significant systems—and you're still using the unsupported system while you do the big bang rewrite, and quite possibly after it fails, so you even in the best case you haven't eliminated the problem you point to with doing an incremental rewrite. So, the basic problem exists either way. Incremental rewrites (strangler or otherwise) prioritizing the highest-risk components for earliest replacement is one plausible risk-mitigation strategy, but the right choice is going to depend on project details. When you eliminate real options because of fake “you can't do that” considerations, you increase the risk of choosing a suboptimal approach because you preemptively discarded the least-bad solution.
You keep using this term and I don't think you know what it means. The Strangler pattern IS an incremental rewrite. You've suggested that it's possible to introduce incremental rewrites to old code and all I was clarifying is that by refactoring code written in an old framework (e.g. PHP 5.6) does nothing to eliminate technical debt (adds to it, in fact). The Strangler pattern is most often used when you want to switch languages (e.g. Java -> Rails) or when an older paradigm doesn't have a straight migration path (e.g. WebForms -> .NET MVC). The new code using the new framework essentially strangles the old code.
For the record, I've advised over 100+ software companies, which I would say about 60%+/- of them are experiencing some type of major rewrite and of those, 9/10 are because they simply can't upgrade an outdated/unsupported/poorly architecture software framework. Trying to refactor an unsupported framework is simply not an option. You (1) either migrate it to the latest (if possible) and refactor over time, (2) strangle it with the new framework, or (3) rewrite it. That's it. Every other topic discussed is simply just one of those but semantically wrapped in some engineering jargon or nuance.
This is all consistent with the examples here: https://paulhammant.com/2013/07/14/legacy-application-strang...
- C++ -> Java Spring
- Powerbuilder/Sybase -> Swing
- VB6 -> .NET
- Java -> Rails
- Java/Swing -> Rails
The real world is FULL of these. I've seen many of them first hand. The rewrites you hear about in the "SV world" are either superfluous CTOs who are misguided into thinking they need to, for example, rewrite their Rails 4.2 app in Node because they think it'll get them more users, or represent real engineering feats that truly "blitzscale" startups entertain to maintain business continuity (Twitter's migration from Rails to Scala comes to mind).
I'm pretty sure I do. But I'm also pretty sure you don't know what the term “Strangler Pattern” means (specifically, that you think it is equivalent to “incremental rewrite” rather than one specific approach to incremental rewrite.)
> The Strangler pattern IS an incremental rewrite.
Yes, but not all incremental rewrites are the Strangler Pattern; that's why my first post in this subthread points out that the choice isn't exclusively between Strangler and big bang rewrite, because incremental rewrites are possible without the Strangler Pattern. In fact, i alos discussed the specific differences that can arise between Strangler and non-strangler incremental rewrites.
> and all I was clarifying is that by refactoring code written in an old framework (e.g. PHP 5.6) does nothing to eliminate technical debt
That's not at all what you said, though it's possibly what you meant, if you were writing very imprecisely. If it is, though, it's odd to the point of non-sequitur as a response to anything I've written because I never suggested refactoring code while retaining an old framework, I suggested an incremental rewrite similar to what is done on Strangler but without (1) implementing a new-system proxy as a first (or, potentially any) step, or (2) adopting an “old code is deleted but never modified in the course of the transition" rule.
> The Strangler pattern is most often used when you want to switch languages (e.g. Java -> Rails) or when an older paradigm doesn't have a straight migration path (e.g. WebForms -> .NET MVC).
Yes, though there is no particular reason that either of those cases require Strangler for incremental replacement.
> For the record, I've advised over 100+ software companies
Good for you, but that's not at all relevant to the discussion.
> You (1) either migrate it to the latest (if possible) and refactor over time, (2) strangle it with the new framework, or (3) rewrite it. That's it.
No, it's not, unless you are using “strangle” much more broadly than the Strangler Pattern, which isn't just an incremental replacement by a particular strategy for incremental replacement characterized most notably by placing a request-intercepting facade in front of the old system.
So why would I use the word "an" as in, "The Strangler Pattern is an incremental rewrite", as opposed to "the"?
> That's not at all what you said, though it's possibly what you meant,
Weird, the following was my first response to you. Shrug...
>Incremental rewrites assume that your framework of choice is still supported. In my experience, I would say 75% of the time someone is considering a rewrite, it's because their framework of choice is out of date, which makes it impossible to modify old code.
FYI - My choice of the word "impossible" was poorly chosen. It's not impossible, it's just stupid.
The original parent was basically asking "if we can't do strangler or big bang for a legacy app what else is there"? You suggested that incremental rewriting is a 3rd option and clarified that a Strangler Pattern is a subset of an incremental rewrite, which I agree. You implied that this meant continual use of an unsupported framework, which I sought to clarify and advise against.
> No, it's not, unless you are using “strangle” much more broadly than the Strangler Pattern, which isn't just an incremental replacement by a particular strategy for incremental replacement
I am indeed.
> characterized most notably by placing a request-intercepting facade in front of the old system.
This is incorrect. The Strangler pattern doesn't necessary mean you strictly write a facade. Furthermore, like all design patterns, they're up for interpretation. What determines the difference between a router, adapter, proxy and a facade? All technically can be used to intercept incoming requests.
Fowler, who popularized the Strangler concept, says[0]:
An alternative route is to gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled.
AND then says
In particular I've noticed a couple of basic strategies that work well.
Which implies that there are various strategies (not just one) that are enacted under the term Strangler Pattern. Hence my previous comment "alternatives discussed here are simply subject to semantics and nuance". You could certainly use Adapters, Routers, Decorators, Proxies, Bridges, etc. for design patterns used in an overall Strangler strategy.
I think we're actually saying the same things, you just seem to prefer to be overly and unncessarily pedantic about your use of the terms Stangler and Facade.
[0] - https://martinfowler.com/bliki/StranglerFigApplication.html
Technically speaking, you can build software on an outdated and unsupported framework, sure. But why on earth would you, for example, continue building your software on PHP 5.6 (EOL Jan 2019) when anytime a CVE is identified, you're basically SOL in being able to patch it.
But once it's too late no migration strategy rids you of the old codebase and it's environment now. With the full rewrite you need to maintain and run it until that's done. With the strangler the old code still runs until it is replaced.
- Compilers/IDEs that only exist on single machines (expensive mainframes, single user VMs, SSH hosts)
- Compilers/IDES that must be air-gapped for any security reason (including CVEs and expired support, not just PII/PCI/confidential-quarantine reasons)
Things start to feel impossible when you have to use a Windows XP VM because it was the last OS Microsoft officially supported the VB6 IDE, who knows how to install and license the ActiveX controls anymore for development (some of the vendors don't even exist anymore to ask), the VM is intentionally air-gapped/quarantined from internet traffic because it is an XP VM so there's now some fun extra steps with virtual folder redirects and worktree-less clones as remotes to get git changes in/out. (If your VM stopped supporting basic copy/paste from the host machine you feel you might go mad. Trying to grep VB6 code in VSCode to make navigation feel somewhat more sane due to scroll delays in the VM feels like its own growing madness as VSCode believes you to be insane using such an arcane language that looks but does not smell like VB.NET.)
Uh, not that that resembles my current situation or anything.
There are similar horror stories from say COBOL devs with airgapped mainframes.