Software engineers hate code
dancowell.com
dancowell.com
They blasted a hole through two walls because it was the quickest way. Then just taped some plastic over the second hole. The project manager said it's still summer and we don't really need that wall at the moment.
I know who will have to fix the wall when the winter comes and I'm not looking forward to it. There's nothing in it for me and I'll get asked why my wall wasn't OK in the first place?!
No more than 6 months later a contractor tried to add a shelf to the unenforced wall. The contractor was never aware of what was in the wall and assumed he could find a stud. What he thought was a stud was actually the stud finder pinging on a water pipe. The construction workers didn't have time to run any tests with stud finders so they just wired everything up at the behest of management and hit the wall a few times with a rubber mallet to see if it held up. The project was behind by a week already due to the plans getting to the job site late.
He pounds in the nails, hits the water pipe, collapses the wall and floods the house. The on staff construction crew lose their jobs, the VP gets a promotion for showing they did something about this egregious error, and they are replaced with contractors.
“The thing that hath been, it is that which shall be; and that which is done is that which shall be done: and there is no new thing under the sun.”
Also "Of making many books there is no end, and much study is a weariness of the flesh" way before it exponentiated to the moon ...
Honourable mention to "For everything there is a season", as sung by The Byrds.
Just kidding, they're lazy.
The framers are locked into the house plans as far as stud distances and special framing members that are specified. County inspections will make sure that's all there.
House construction is far, far more standardized and strict that programmers coding crap code. It's not a good analogy.
In many ways the builders are even worse because they can usually hide the faults with caulk and paint if you want. Problem becomes evident when things start to leak or rot. That can be later but been causing damage in the meantime.
https://www.youtube.com/watch?v=FfEX6GUxfZM
Here's an example of an arguably good builder who had one team install a in-wall tank toilet, then another team drills right through the tank causing a leak. The repair work essentially meant ripping out everything and doing it over, even though there should have been mitigations in place to prevent it and having the plans/documentation available to everyone.
While construction has more standardization, planning, and inspections, it still relies on implementation and is going to have failures.
> House construction is far, far more standardized and strict that programmers coding crap code. It's not a good analogy.
Then, I agree on the point of someone else blasting a hole in your house.
Years later I spoke with an ISP about an install and they proposed exactly the same thing. I think people do this a lot.
Both are true here. People don't get away with quite that level of nonsense with buildings because even laypeople can see all the essentials, why it would be bad, who to blame if it did happen, etc. So it never happens and doesn't need any kind of illustraing analogy to explain it to anyone.
That DOES happen all the time in software and it's invisible to almost everyone including your own bosses, and does need some kind of analogy to illustrate what's wrong about it, and how wrong, and the nature of the wrongness.
Any other example that actually ever happens, would just be some other equally opaque phenomenon from some other esoteric field, and would by definition be useless as an analogy.
The whole point is so that anyone can see it and conclude "that would be crazy, outrageous, intolerable". Of course it never actually happens.
To me writing a story comes closer to what writing software is. I can't just get an overview of their plans and know right away where I should be connecting the new "room" I want to build. Instead I need to understand what story the author was trying to get across and make my insertion in a way that makes sense with what was there before.
Someone put fake shutters on the sides of the windows - the wrong size at that. This was accepted as a good solution because no one tried to close them. Would be impossible anyway, because there are no hinges.
This happens more often than it might seem.
Once you're a senior dev, you start to look at the project holistically, adding your input to the design in order to ensure that the project serves its objectives. The code is still important, sure, but the success of the project is more important (and that can sometimes involve some short-term ugly changes).
If there's a business need for a hole in the side of the house covered with plastic that will probably be there for the next year, it's expected that you've all had many meetings discussing this modification and its necessity. By the time the code changes come, you've either long agreed with the reasoning, or have agreed to hold your tongue, or have started shopping out your resume.
Regardless, by the point this code is written, you'll already be expecting it and won't be blaming the dev who wrote it.
What you are describing happens in an ideal world of software development. It exists but it's rare.
When it wasn't, I shopped my resume around.
Except that I've met lots of engineers who were the opposite. They hate greenfield projects and prefer maintaining existing code. I noticed this broad division of personality early in my career. And it's a great thing -- both sorts of engineers are critical to a successful project.
Nothing is universal, but I think often the latter.
I just wanted to use the tool without any effort, but I have to understand the interface and patch the tool to make it work again. I'm not complaining.
Just wanted to add a data point for the former part of your comment.
That code also has been written by a human, and it's disrespectful to approach somebody else's work with this prejudice.
It might not be good code, by our standards, but who knows how it was written. Was the author under stress, under time obligations, or happy, or something else?
Also, being able to approach somebody else's code and work with it is humbling and deeply educative for me. You get a (free) passage to a wisdom of a person or a group. That's priceless.
Not only has it been written by a human, but one should also remember that they're missing the context in which the code was written. Was the author stressed about their latest children being born? Fearful of possible layoffs? Under time pressure? Tired?
I've definitely written code myself that came to bite me later, and I marveled at how badly written that was.
We're all human, which, among others, means that we're all inconsistent to a large degree.
Too many developers remain hyper committed to shiny object syndrome, or wanting to relearn lessons of the past as if they are the first to ever lay eyes on it.
No code is perfect, and lots can be learned even from a well intentioned approach gone wrong.
It’s far easier learning from other folks and their rattlesnake bites instead of insisting of recollecting that baggage yourself, first.
Greenfield is still useful, but too often it has to do with preference and interpretation or a lack of willingness to do so.
Small piece of advice: it is much better to learn how to take criticism of your code as exactly that - it's criticism of your code. People will always be unkind when reviewing code. Even if it's the best code ever written someone will have a different opinion and express it in a seemingly unkind way. It's easy to be unkind in code reviews because it's _code_ being reviewed and NOT a person with feelings.
Like you said, a different human (the more junior version of you) wrote that code, not you. It's not a criticism of you no matter how much your brain tries to tell you it is. People will never stop saying "who the fuck wrote this shit!?" and you will hear that phrase until the end of time. It's still not personal when people yell that in frustration.
For example, I have learned that Eastern Europeans tend to be absolutely brutal in code reviews. Everything will be nitpicked and the most benign design decisions you've made will be questioned. They make you feel stupid and inadequate but that's your own feelings. It's more likely you're dealing with a direct and to-the-point perfectionist who can see the code beyond your PR and how your code fits in with the rest of the application. They're not the greatest at communicating this context and their comments seem like personal attacks without that context.
I love the Eastern European approach; blunt and to the point. The French are also good for this. We anglos are so sensitive, and so afraid of offending, that you end up with a silly inefficient dance. “Have you considered? Perhaps if we tried this approach…”. No! Give me “this won’t work, do it this way” every time!
This is not about style (kind or to the point, or in any style), but again about keeping in mind that you're talking with a human being.
You can say "This doesn't work (because of this), so do it this way", in thousand different intonations, and you can add the underlying message in anywhere on the spectrum of "this works better for our case" to "you're a triple distilled idiot".
This is not about being sensitive or having a skin, this is about mutual respect in a professional setting. When you do your to the point, brutal criticism, you cannot hurt the other party if you keep in mind that you are talking with a human being.
During my M.Sc. jury, one member called me a liar, and my professor had to intervene politely yet forcefully, and in my Ph.D. jury one member flat out insulted me, yet I defended myself politely yet forcefully.
They could have said the same thing, with the same words, in a much politer tone, but they preferred to attack me directly instead of attacking my work, so there's that.
I tend to give direct criticisms as well, yet I do my best to make sure that it's plain and simple I'm trying to guide and it's about the work, not about the person.
Because, I do not do anything to anyone which I don't want to experience myself.
I want to reiterate you will never be in a situation where everyone is respectful even if it's a professional setting. You also can't control what other people do so there's no point expecting people to give criticism respectfully or trying to teach people how to do so - you can't correct someone's behavior if they don't want to change it.
Every time I catch myself wanting to say "people should act like xyz" I stop myself and think "I will react a different way when someone acts like xyz" instead if I'm not happy with how the situation went. Especially if I've already expressed my feelings and someone has ignored them and still acts in a way I don't like.
I think I misrepresented myself. I don't expect anything from anyone, in any setting, or I don't intend to change anyone. Who am I to expect to change someone?
I just wanted to say that, I behave the way which I want to be treated. I ask myself, "what would I feel if that's done to me", and if I don't like the answer, I change myself, and try to better myself step by step.
Actually "The Four Agreements" by Don Miguel Ruiz sums up pretty well how I live this life. It's a surprisingly thin yet dense book. I recommend it wholeheartedly.
Looks like we try to do the same things with slightly different methodologies. My hat's off to you dear sage internet stranger.
Wish a peaceful future awaits you.
Startups naturally thrive on an early team that loves building new things, but the flip side is that they rarely seem to go back to fix existing stuff. That leaves a gap for people like me, who prefer to take something that works poorly and make it bulletproof. For some reason this quite often seems to involve db access patterns but there's a fair bit of algorithms work too. Usually the "building new things" engineers are quite happy to let me take over the investigation into why the 150 line SQL query is slow, so it's a win-win.
Understanding greenfield usually means iteration, a small amount of conceptual architecture and organization before dialing into the matrix to go full Neo can go a long, long way.
Specifically, it’s easier for startups to create productive new beginners in their codebases.
I am also an amateur fiction writer, and I notice the same thing in it. I like planning, but the first draft writing is a chore to me. An effort to vomit all my ideas as words. Then, I love rewriting. Revisiting what I wrote and improving it, fixing it, perfecting it.
It’s fun early in career, but gets boring fast IMO. These are good things to have junior and mid eng cut their teeth on.
If people did learn those intentions we wouldn't have the problems we have.
I inherited a project like this once as the third generation to take it over. The (evidently, I never met them) well trained and experienced first developers laid down a foundation using a textbook OO style with a few little tricks needed to make that work in a language where OOP support was bolted on. The code was not perfect, but it was professional work and I would put it around the 80th percentile in terms of code quality.
These tricks caused the virtues of that foundation to be overlooked entirely, and so the project was essentially reimplemented in spaghetti style over time with various ad hoc stovepipes to pass data back and forth between the two sides when the original had functionality that wasn't understood well enough to be reimplemented.
The importance of training, communication, and retention can not be overstated.
Me too, mostly because I hate starting a project form scratch, it's something I'm not particularly good at. Like you, I love rewriting, I love improving things, making them better, faster, or easier to understand.
The notion that software engineers hate code is wrong, depending on who you consider a software engineer. SREs, systems administrators, operations people, they hate software with a passion. Developer (who in my mind are software engineers) freaking love code and will write more code to cover up problems which should have been address by revisiting designs and earlier assumption. I've meet with developers who's understanding of IT outside the realm of development have been so extremely poor that they frequently reinvent basic webserver or database feature, but badly.
Currently I'm working on a Django project, in the back of my mind I always have a notion that if something requires more than 20 - 30 lines of code, then it's either already a feature of the framework, or I'm doing something terribly wrong. That rules has yet to fail me.
Some authors really need an editor.
Not a rebuttal just an addition. All these things are true.
I don't want level 3 support or on-call rotation, just established software that needs diagnostics, bug fixes and upgrades.
I guess you just have to apply and ask in the interview.
I recommended “Working With Legacy Code” and he read it.
His first job so great experience. I guess be underpaid is the answer.
I have a friend who does migration tools that help businesses move their data from ancient versions of software that the big players don't support any more. It's a niche, but it's a niche that banking/insurance/etc customers are willing to pay for. I've applied for jobs like this and never even made it to the screening call. When I asked how she got the job, it was through a former colleague of hers. I actually interviewed with that colleague at a different company several years prior and I heard from her that his impression of me was that I would be bored in the role. So I think it's really important to emphasize to the hiring manager that it is exactly what you are looking for, especially if you have a background in more "hip" companies.
If the "boring" companies don't call you back, my recommendation would still be to look for B2B software, or an internal/tooling-focused team where your customers are your colleagues. It has to be on a product that is already profitable (or seen internally as valuable) so that you don't get pressured into adding new features or having the whole product killed from under you. Ideally a company that is not on the IPO track. There are some maintenance roles to be found in these "hip" companies, but they're rare, and I think you have to accept that you might only be able to fly under the radar for a year or two until an ambitious executive comes along who wants to reorg the department into a boondoggle production line.
1. Only working on greenfield means you don't want to learn from others or have difficulty doing it. Lots of knowledge is expressed in the code you refuse to read and you need humility and patience with your own ignorance.
2. Only reading code is insufficient as you are not only a scholar of the code base, but a maintainer and developer who must extend and adapt it.
Working with a code base is more like a conversation. There's a back and forth, though not symmetrical. (1) is someone who monologues. (2) is an audience member only.
But for the other 85+% of people in the profession its basically a game of AdLibs. Fill in the blank. That's why there is such heavy reliance on things like frameworks, dependencies, and Invented Here Syndrome (people who don't trust their own or teammates code). That's why you have expert beginners, because you can get really masterful at coloring with crayons.
Try to take any of this color by numbers nonsense away and see immediate full on emotional apocalypse. It's not a choice. The number one priority is to retain employment, and if your employer has failed to provide you the preparation or environment necessary to perform with confidence this is what you get.
This is so true. I work at a fairly small organization (~35 devs). Large enough to see roles form but small enough that I can tangibly see everyone's contributions. I work SRE, so I see lots of code from all the teams and ultimately talk to almost all the devs.
In our impressive enterprise-level codebase (our SaaS product has average annual subscriptions in the range of $1.5M per customer), there are probably 3 developers that wrote 80% of the codebase. I can name them all and call them up. But I said we have 35 devs in the organization. So what do they do? Mostly little things. Flipping a conditional or making a slightly more explicit test case to fix an unexpected edge case we experienced.
Most of these 32 other engineers are solving simple problems, basic refactoring, and so forth. Not too different from the adlib, fill-in-the-blank analogy you provide.
Also, to be clear, it isn't like we hired 3 seniors and expect them to write all the code. These are just groups people fall into naturally.
The reality is most software engineers have only ever done the adlib style work and have built an entire career around never solving any real problems. They know just enough syntax in a language to solve the little problems and with software, there is a near infinite supply of these little problems.
I'm curious if the ~three devs that "wrote 80% of the codebase" mutually recognize one another? i.e. do you think they think of themselves as in a "writes a lot of code" category along with the other ~two devs? Do they think of themselves as "writes a lot of code"? Does management recognize this and/or plan around it?
I've sometimes been one of the "people writing a lot of code" in a small org. When I'm in that mode/context, I find myself daily appreciating the types of contributions made by the larger body of folks less deep in the woods. They do a different and complementary type of work, and I've found both types bring something valuable (and different) to the codebase.
Being a manager/lead developer today, I enjoy the people doing small work. I just let them pick tasks as they wish, and it takes longer, but they eventually pull through.
The problem is just that they're a bit unpredictable, so if something has a tight deadline, we gotta assign to someone else.
Worse than that are the people who work their ass off, but just aren't good at it. I would rather work with people who prefer to not do work. The output is generally the same, but the people who work their asses off cannot see the difference and believe they somehow deserve more.
In my career the single biggest failure that I see repeated EVERYWHERE is confusion about what software really is. There is only one purpose to any of this: automation. In that mantra actually not doing work is supremely desirable so long as the output meets expectations.
I think the biggest risk with the people who work their asses off more than they should (doesn't matter if it's good or bad code) is that they end up with a non-sustainable output. Productivity is artificially high, and often achieved by overtime, working weekends, and overworking themselves. Eventually that leads to burnout, lower quality, and it becomes hard to replace. When it's people working too much and delivering bad code it's probably not as bad, though.
At my current company we have someone in this situation, with work affecting their day-to-day stress levels very much, to the point of damaging their personal relationships. Their manager is young and inexperienced, and doesn't see the issue, but he was flagged by HR and I hope something gets done.
How does a non-tech role sift through the 32 devs who did 10YOE flipping bools vs. the 3 that built the entire system? They'd look similar on paper and in superficial conversation.
The issue is that in many companies there is constant pressure from management to "innovate" by building new features or new products, even if they are of no particular utility. New stuff will bring new customers (or new subscriptions from old customers) and that will either make you personally rich, or at least look good on your CV so you can leverage the experience into the next job where you can get rich. And if the project fails, it still leaves you looking more valuable to the top brass than the people who quietly kept the systems running, because you took risks and had big ideas and showed a growth mindset.
I didn't really notice this at the start of my career, but once you see the pattern it makes so many jobs incredibly depressing. People just keep building more and more crap to advance their careers and enrich the shareholders, even when very little of it is actually useful. It's even more depressing in B2C where it becomes apparent that a large part of the money comes from exploiting individuals who simply get a buzz from buying new stuff, no matter what that stuff does or how good it is.
Personally, I am much more happy to work on an established system that already has real customers that value what it does. The phase of throwing huge amounts of crap at the wall to see what sticks is over, so what remains is an actually-useful piece of software that's full of bugs and superfluous functionality that the "innovators" (comprised of both good and bad developers) left behind. I think it's worth it to have good engineers dedicated to this role of taking the mess and refining it into the best version of the thing that customers actually want. Leaving that task to the bad developers will just result in the system eventually collapsing, which might not matter to the entrepreneurial types who only care about growth, but it does matter to the customers who were getting value from it.
Most if us aren't that but we just continue about our business developing new features and squashing bugs. I like greenfield projects but the choices and freedom they present come with a lot of non-coding and missing infrastructure.
A nice brownfield project lets me check in code, see it auto built into the dev environment and have QA feedback the next day.
Funnily enough, personally, I can handle badly designed projects as long as I was part of the team designing it. At least then I understand it enough that changing things isn't too much of a pain, and I can pretend my refactors will eventually make the codebase look good (or if that fails, I can write enough documentation to at least mitigate the problem). It's a pain in the ass for new maintainers though.
But goddamn, working in a good established project where other people have done the hard work of figuring out non-coding project management stuff is a fricken breeze. No arguments there.
Earlier in my career, I was stereotyped as a "maintainer", but now I've been a self-employed indie developer for years, so all of the code is mine.
Once your own code gets to be 5 years old or so, it's almost like maintaining someone else's code. It's like, why in the world did I write that before??
I guess my focus has always been on the product and the user experience, so I don't make the code itself central to my work and my identity. The code is not the product.
Even in well established code bases there can always be "green" new pieces. These are choices to make: should we refactor the code, should we fix the old code, should we rewrite pieces of it. There is no one answer that always works. The problem is when you look at something and your answer is always a rewrite just because you didn't write the original code. I've seen people/team rewrite perfectly good code for no business reason, just because they want to.
In terms of code written by others, sometimes that's a thing of beauty, a piece of art. Sometimes it's just garbage. I'm sure we've all seen some code where we go wow, this is incredible. And then we've all seen code where we go wtf. I don't really see the need to dig too deep there, what's great is great and what's garbage is garbage and then a lot is somewhere in the middle. I sometimes look at code I've written in the past and I think it's really great; and sometimes I go wtf. Usually the latter is where I was not familiar enough with the problem space or the code base I was working in. Questions to ask are: does it work? what's the cost of maintenance? Ugly code that works well, doesn't need to be touched, can be fine.
Repairing / fixing / retrofitting an old code base always has it's wonders. I've seen much code in my life that was in a bad shape and my approach for fixing was seldomly the same, it was always tailored to the team and situation.
To rephrase from Tolstoy: "Good code bases are all alike, every bad code base is bad in it's own way."
I’ve had 8 jobs over 25 years and over that time, I’ve mostly done green field work, including at my first job out of college.
I’m not even sure I have the skillset to modify a huge existing code base.
Even at one job where there was a large codebase that I was “modifying”, it was mostly to add new CRUD features to an API or website and I did my own vertical “slice” that didn’t involve modifying existing code.
It seems like this is a good opportunity for LLMs…
A related problem is that "maintenance" is conflated with editing code, but IME some languages especially """bug free""" would prefer you rewrite every time you make a change rather than edit things, because the code becomes impossible to understand after it has more than one author.
Start your project on the right foot by finding that disgruntled senior engineer and giving them ample time for design and research.
I also spend a lot of time repairing and maintaining old cars, houses and fixing up stuff to be better and stronger than it was when I found it.
It’s very difficult for me to leave well enough alone. If it ain’t broke, it could certainly be better!
"Things You Should Never Do, Part I
"They did it by making the single worst strategic mistake that any software company can make:
They decided to rewrite the code from scratch."
[1] (Emphasis his.)
Where I'm 100% sure is that consultants hate code. I've never ever seen one recommending the reuse of existing code - not once.
And it's understandable: They have nothing to gain from recommending reuse of existing code.
In the best case the code is good and everything else goes well and the client saves some money. If the consultant can not pull this of repeatedly their benefit will still be limited. The praise will be with the programers.
On the other hand, if the old code is bad or anything else goes wrong everyone will blame the consultant for the recommendation.
For the consultant it's a risk-return tradeoff that just always favors a rewrite from scratch.
[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...
To utilize all those benefits you can't rewrite everything at once from scratch. You need to do it incrementally.
On incremental improvements you should be able to stop what you are doing within a week or two and be ok with leaving the code like that for a long time.
Incremental improvements require more skill. The sorts of skills that make you a better developer. So even though it’s more difficult, it’s much more valuable to all parties for you to attempt it.
The only reason for a rewrite is when you have a language or framework that’s dead, and there are often other dynamics that put you in that spot in the first place.
There are a bunch of common mistakes people make with trees that take years off their life expectancy. The worst guarantee the tree will be dead in twenty years (and potentially dangerous before that). Software has similar phenomenon, on a much shorter time scale.
My knee jerk reaction to rewrites is so negative I actually would like some counter evidence on when it worked out.
[1]https://www.postgresql.org/message-id/4658520e-5cd0-6242-e54...
The common factor is doing it gradually, basically page by page. There is no big switchover, new code goes live as it is ready. We have some very competent people with long tenures which surely helps.
I'm not against rewrites, sometimes the company grew so much that you need to start from scratch and rethink given your current knowledge. Or you started with a low fidelity prototype and you rewrite to create a production ready feature, without the crufts in the design you had when you started.
Edit: refactoring is usually considered to be done on existing code base. I really mean thowing a whole component away and re-doing it from scratch. The components need to be small enough though, that you can deliver them to production within 2-5 months.
Code gets re-written from scratch all the time, for perfectly legitimate reasons. As even Joel Spolsky knows. He just meant that you should be aware of the trade-off, and that most of the time, you probably don't want to re-write from scratch.
But "Never do X" sounds much catchier. And succeeded in getting the article shared and quoted infinitely and endlessly, to this very day.
Even though everyone knows it isn't, you know, actually literally true. Even Joel.
Yea, mechanics love to complain about metric and imperial sizes and when they don’t thread in correctly, but at the end of the day a good mechanic loves seeing a smooth running car.
Similarly, a good software engineer loves it when they have a smooth running service. Updates work without hiccup and the system can be inspected to see how things are running. Having clean and maintainable code is directly proportional to the ease and pleasure of work on this service.
I believe in quality over quantity and I also believe that beautiful code exists. My hatred of code comes from lazy or rushed implementations, which can and should be improved over time.
LGTM culture is a product of bad management and should not be used as an excuse to ship lazy code or code written by people who hate what they do.
A good engineer should take pride in their work, even on the bad days. And even when the product itself isn’t what they would personally want.
What are you referring to specifically here? A culture where nobody is actually doing reviews and just LGTMing everything?
To make it more fair, code you wrote yourself doesn't count, because you aren't an objective observer -- and more importantly, have a different relationship to it making it easier to work with since you wrote it, and often to your own preferences.
I think I agree with you, it's just that... the actual way software is written doesn't seem to allow for much clean and maintainable and easily extensible code to be written, so I'm not sure how much it matters practically.
I disagree strongly that “the actual way software is written …”, though it does feel like trying to write clean code is an uphill battle sometimes.
Back to the shop analogy. Cleaning up oil spots and maintaining well polished tools requires time and care.
https://www.goodreads.com/quotes/57688-i-hate-writing-i-love...
While the article resonates with me alot, I would like to, in the best spirit of the article, propose an addendum to that line:
When modifying existing code, do a very careful cost-benefit analysis; On the one side is the cost of a rebuild. On the other side is the projected cost of keeping this thing and maintaining it, not just for this change, but for changes in the forseeable future.
I realise that this is essentially an impossible requirement. We cannot forsee the future. But: We can make predictions. And when the predictions say, that, forseeably, the company will lose money down the line because we waited to long for a rebuild, it may be time to pitch that to whoever allocates resources.
Because, dragging a legacy-system along incurs it's own set of costs. This is especially true when it's not just maintained, but modified and extended. And many of those costs have a nasty tendency to remain hidden until they suddenly don't, and at that point, people often already expended inordinate amounts of resources on them.
So yeah, the first instinct should be: Use what already exists. But check the costs of doing so. Premature rebuilds are a waste of resources. And so is holding on to legacy systems past their expiration date.
By the estimates of another team, it will take 2-3 months to build a wrapper around their codebase (it is that entangled) and throw that in EC2. The whole project will become infested with that codebase and those issues because as we all know, a “temporary fix” is never temporary. The codebase doesn’t cover anywhere near what we have in mind for features and extensibility is … yeah.
That’s a common fallacy of programming: the existing code is convoluted and hard ti maintain, but the new code I would replace it with will be much better and much cheaper to maintain.
Yes, the cost of the rebuild itself, PLUS it's own maintenance, fit with the rest of the codebase, extensibility and so on, must be considered in that equation.
I glanced over that, because usually the rationale for a good rebuild is an improvement regarding exactly these metrics.
Assumptions and Environment change. New becomes old. And old has maintenance cost.
Only developers who love greenfield or need a new framework on the CV would suggest a company could lose money by not rebuilding.
If the developers are not competent enough to write maintainable code or maintain existing code, then you will have exactly the same difficulties after the rebuild.
If they are competent enough to write maintainable code and maintain existing code, then you have no need for a rebuild. Just adapt and extend the existing code to meet the new requirements.
Or if the old system simply doesn't work with modern environments.
Or if it depends on long abandoned frameworks.
Or if the business grows but the old implementation scales badly or not at all.
Or if it depends on components that incur licensing fees that become prohibitively expensive when it's scaled up.
Or if there are other legacy systems on different technical baselines that it could work with better after being rebuilt on the same base.
Or if its tech simply requires more maintenance than an alternative, thus binding dev resources the company could otherwise use more productively.
There are alot of reasons why maintaining an old system may be an undesireable move in the long run, that have exactly zero to do with the competence of the developers involved.
I’ve met quite a few “unlucky” people in my life and what nearly all of them had in common was the inability to connect their actions and inactions to consequences. They were blindsided by predictable outcomes over, and over, and over again. Eventually the people who could actually help you get tired of your drama and move on, and then you’re left in an echo chamber where your narrative makes sense.
I’ve spent a lot of time on this job thinking about the thoughts and plans of the people who have left. If our vertical were more compelling I believe we could make a better product with the people who have left than with the people who stayed. Even considering the lack of depth in domain knowledge. There’s a lot of things that don’t change because they were the right thing to do ten years ago. That’s an explanation for how we arrived here, not a reason to stay.
Sometimes they are. I've seen several applications that were successful for many years but eventually had to be rewritten because some vital dependency was no longer viable.
In web development we had an early generation of web apps that used plugins like Flash or Java to do things. Fast forward five years and those plugins have been brutally killed off by the browser developers. However other web technologies have become viable alternatives for some of those things. That's a big rewrite.
Some programming languages have had big jumps that weren't entirely compatible. Python 2 to Python 3 is an obvious example that took years but eventually resulted in not only Python 2 no longer being supported but some libraries never being updated to support Python 3 and others being created to provide similar functionality. In this case many of the direct code changes could be automated but you can't automate swapping out each obsolete library for a replacement with a similar purpose but a different API. And maybe you wouldn't want to because in the 5 or 10 years since you built the last version new ideas have come along and you're better off adopting them instead since you have to make a big change anyway.
But this is also a very different scenario than the grandparent example, where the cost of maintenance due to bad code quality is supposed to justify a ground-up rewrite. In the Flash example there is a clear business case for porting, even though it is understood the porting will expensive.
The fallacy is believing a ground-up rewrite will lead to more maintainable code. This is just developers deluding themselves. There is no reason to think a ground-up rewrite of existing working code will lead to better and more maintainable code.
As with almost every argument in this area that depends very much on context.
A substantial rewrite might be an opportunity to use better tools and improved techniques that have been developed since the original was written. In some cases that could represent a huge improvement in things we actually care about like developer productivity and the quality and performance of the product. Importantly this doesn't imply anything was done wrong or any bad decisions were made by the developers of the original product or the people who have maintained it so far. It's just that we work in a fast-moving industry and sometimes even a few years can see some big improvements in what the available technologies can do.
A development team starting with a relatively clean slate can take into account all the knowledge and feedback received over the lifetime of the existing system. Maybe it was too expensive to make use of those insights while evolving the original system but a new version can take advantage of them. Again that can represent big gains in areas that we care about.
It's often observed that a big rewrite risks losing a lot of small improvements and fixes that have accumulated over the lifetime of the existing system and of course that's true. However it's also true that a big rewrite can avoid a lot of existing design problems or get rid of long-standing bugs that no-one was ever going to get around to fixing.
I've seen big rewrites that didn't end well. But I've also seen big rewrites that were done for sensible reasons and had very positive outcomes. And I've also seen things that should have been rewritten but weren't and instead became a drag on everything. There is no universal rule here.
Can you give an example? I have a hard time imagining what kind of techniques cannot be applied to an existing code base with some adaption, but require the code to be written from scratch.
Another is when your platform evolves and forces the issue like the web plugins being replaced by new web standards that we were talking about before. Here you might not need to rewrite your entire application but you probably are forced to rewrite the affected parts and possibly significantly change the software architecture around them. A related example is if you previously wrote your application targeting a specific environment and now want to support multiple environments but the operating systems or frameworks or other dependencies follow very different conventions that impose some constraints on your software design.
This is a hill I will die upon: The people who don't deserve rewrites ask for them, early and often. The people who deserve a rewrite rarely mention them, and in fact they likely already have done it, bit by bit as a sibling comment mentioned by way of example. I will say that I've often surprised myself with features I never would have hoped for on the refactoring road. Things that would have been entire Epics become six weeks of work, sometimes less.
Refactoring is the Ship of Theseus scenario. You replace the ship bit by bit, until it's both a new ship and the same ship. Yes, it's a titanic pain in the ass, but it's also atonement for your past bad decisions. Which are heavily populated with cut corners you will be able to spot the next time.
All worthwhile learning is effortful, and greenfield is the lowest effort path to anything. You don't learn much from greenfield except why greenfield is not a panacea.
The calculus of a project that has been running for 10-20 years is often different.
I've explained why - and why in some cases the approach you advocate is literally impossible - in my other comments in this discussion, which I invite you to read if you haven't already.
That depends entirely on the reasons why they ask for a rewrite.
"I like this tech better", "I don't want to work with this tech", "This new tech is shinier": I agree with you, those are not solid engineering reasons.
"This doesn't interface well with the rest of the system because...", "This is going to cost us in the future because, ...", "This won't scale well because..."; I strongly disagree.
You may notice that the operative difference here is the term "because". If someone can give a quantifieable, technical, verifieable reason for a rebuild, then management should at least hear the guy out. They can still say no. But then the engineer did his job, and if the whole show goes haywire later because the 14 year old Java backend fails to scale up and the company loses money over that, nobody can say he didn't warn them.
If an engineer propose that the only way to solve scalability problems is to rewrite everything from the ground up, it just tell you they haven't been able to identify the root cause of the problem. The rewrite will probably end up having he same problem.
However all of these are systemic issues and once you've picked the low-hanging fruit you can still be left with systemic problems that are not concentrated in one place but spread throughout your code. Eventually your profiler curve is nearly flat but your JavaScript or Python code still isn't going as fast as C or Rust. Eventually you think you've patched all of your injection points but if you're using manual string concatenation to build your SQL queries you'll probably keep missing others. And eventually you run out of places to add caches and load balancers and it turns out that your existing data storage model is fundamentally limited and needs to be replaced.
In each of these cases you may end up needing to rewrite a whole section of your application or a whole service in your distributed system because you can no longer paper over the cracks. Fortunately it happens relatively rarely but it certainly does happen!
Some trivial applications are basically just glue between frameworks, but most non-trivial application of value will have lots of code which is not tied to any particular framework, and often this is the most valuable part of the application.
Scalability is improved by identifying the bottlenecks and improve the design at those core points - not by rewriting everything from scratch.
I guess moving to a different programming language is a case where you literately have to touch all the code, but even then code can often be ported semi-mechanically instead of starting from scratch.
Granted, that concept can and should be introduced into old codebases. Last year my team successfully warded off the Sirens of Rewrite by just doing the hard work of extracting all of the dead framework calls and then NOT adding them back in drag-and-drop style, but properly exposing them through interfaces that don’t require everything to know everything about the particular replacement framework we used.
"If" is the operative term here. "If" it is well designed, then there is usually not alot of reason to scrap it in the first place.
It is even likely the new version will be worse, since it wont be developed incrementally, and that harsh deadlines will be imposed when the organization realize they can't evolve the product as long as the ground-up rewrite is underway.
That doesn't follow for me, sorry.
A rewrite, as I understand and use the term, doesn't mean transpiling what exist to, say another language or to another framework. The old version is essentially just a very detailed and testable list of functional requirements; everything the old thing can do (as long as that functionality is still actually useful), the new thing must be able to do as well.
How this functionality is implemented in the rewritten version, and how it's internals are designed, is entirely up to the rebuild. The way functionality is implemented in the predecessor does not necessarily determine how it is implemented in the new version.
Why assume the same developers would be doing the rewrite as the original? Maybe the reason for the rewrite is because the original is hopeless and most of the people who worked on it are no longer around.
Also everything usrbinbash said in a sibling comment - but if something like a changing environment forces a big rewrite of an otherwise successful code base then having the original developers still around can dramatically increase the chances of success IME.
Why are the current developers not able to maintain code they didn't write themselves? Is it because the new developers are less experienced, or because the organization culture have encourage writing convoluted, idiosyncratic and badly documented code? Whatever the reason, the root problem is certainly not solved by rewriting the code base from scratch, since you will just have the same problem next time there have been a few replacements.
The assumption that a big rewrite would be too expensive and end up with the same problems is itself quite dangerous. Some of those groups I mentioned knew very well they had a pile of junk but management had apparently read the usual advocacy about how Big Rewrites Are Bad and stubbornly insisted on adapting the existing code instead of recognising that it should be written off. They spent far more time and money on the updates than it would have taken to do a clean rewrite. And then they got into this kind of sunk cost fallacy where because they'd spent months doing what should have been weeks of work once they then became even more attached to the flawed code and kept repeating the same mistake.
Of course this claim assumes you can reliably estimate the time and cost of the rewrite and the claimed improved productivity after the rewrite.
It is still unclear to me what kind of improvements cannot be applied to an existing code base through refactoring and gradual improvements, but requires all the code to be written from scratch.
I almost preempted that counter-argument in my previous comment. :)
Some software development is 90% research and 10% development and it's true that you never really know how long it's going to take and how well it will work until it's almost done anyway. But the tar pits I'm talking about were not that kind of software development. Most of the cases I'm thinking of were the stereotype over-engineered "enterprise" code that had become bloated and excessively interconnected. Others were "clever" code where someone had tried some fancy design patterns or data structures or algorithms, typically with a severe YAGNI complex as well. Either way making simple changes required many times the effort it should. And yet a drop-in replacement for the whole system would have been a low risk project, with very predictable work required that could be done by any mid-senior developer on the relevant team, taking a fraction of the time.
That's going to be very difficult, especially when you then also have to consider that the changes you could do could REDUCE the cost of maintaining it. If you're only ever appending code, then the cost to maintain can quickly skyrocket. If you're also diligent about removing the features that you don't need, there should be no significant difference at the limit.
Basically, assuming you are properly maintaining the existing system, you should be continually refactoring it to be what you would build if you started from scratch. In that view, the cost of maintaining it will be identical, and the only cost that matters is the cost of bringing the existing system into alignment with what you would build now.
Any business that employs software developers knows exactly how much it costs to employ them. If that math doesn’t work then the software developers loose their jobs. This basically establishes a baseline for “value” that must be delivered, at minimum.
Above the baseline, software devs are left to their own devices: There will be no hard accounting of weather development effort is “worth it”, rather it’ll mostly be about whether people feel like it was. (This is sort of the origin of “too much money spoils things”.) As long as the effort doesn’t destroy everything, it can be argued that it was a success!
So, the “hidden” costs are actually “bearable costs” because the business is just fine. The costs are “revealed” only when they’re no longer bearable. Ironically, well designed software “hides” the costs much longer than poorly designed, so what is missing is that we don’t have a great way of assigning value to in-house developed software that “just works”. Developers don’t even really think about it much because they get paid for writing code.
I think that’s why the advice…
> Use what already exists
… becomes very hard, in practice, for devs to follow, even though it’s excellent advice. It’s a problem for management to solve.
Stack overflow is probably another place I differ from other engineers. I'll use it to discover patterns I'm not aware of, but I'm much more inclined to actually Ctrl+click and look at how a thing is implemented and it's sibling methods. Of course, you need well put together local configuration to do all that. I'm always looking for ways to keep my debugger in-tact, even when dealing with things like secret storage on a zero trust network. I use flags a lot for this that let me use mock-local responses.
Then again, I work on infrastructure stuff. The kind of applications I work on have to exist for a long time because of internal contracts and dependencies. Maybe this piece is more aimed at product SWEs.
Looking through library code instead googling can be incredibly productive and polishes your code reading skills which are very important.
What I hate is inconsistency. Inconsistency is what makes code intolerable to work with, because changing it becomes so much harder.
Consistency, in the way I mean it, does not mean DRY or over-abstraction. What I mean is, pick a mindset or design philosophy and stick to it. Don’t randomly switch between exceptions and returning errors. Don’t over-abstract some areas early on and then spaghetti code other areas. Have some consistency in how you do this.
For example, have a rough standard for when something is X or Y. Either accept spaghetti code for areas and keep things uncoupled as much as possible (my preferred), or have some concept of abstraction you apply to new layers. Just rough examples.
If it turns out you did it wrong, which is likely, then it is relatively easy to reason about a change. But as soon as you lose the consistency then it becomes a nightmare. Don’t have special snowflakes in your code.
The last thing I’ll write is.. sometimes the over-generalization this article makes is used as a weapon to justify sunk cost fallacy. Sometimes throwing away a part of your codebase and starting from scratch is the best thing to do. But you should work with it for a bit to understand the code before doing so.
Sometimes, they even are able to create codes that match such abstractions.
Sometimes, those abstractions are now flawed.
Sometimes, those abstractions are properly tooled.
Sometimes, the rest of the team also understands those abstractions.
The conjonction of all these rarely happen.
A while ago I realized that the biggest thing that I like about software development is the simplicity within it's complexity.
What I mean by this is; software either works or it doesn't. In many other professions this is not the case.
So while there are many things that could be done better and more efficiently, at the end of the day, your code either works or it doesn't.
So simple, but yet, so ruthless.
A good developer will write code that passes many of these scenarios, but even the best will miss some. And an inexperienced developer might write code which passes some basic scenarios.
So, “this code works” is really a range, not a binary condition.
In the context of this article and this thread, I was just reminded about this approach.
Hating or loving software development, other peoples code, or approach is sort of meaningless.
At the end of the day, we all get paid to build software and it is our job to make the software functional.
As long as the code works and does what it is supposed to do, thats all that matters.
You don't use threads in your software, do you?
Using the relationship analogy, when you can hear something small from your partner and know what they are thinking, understand what they may do next it feels effortless.
When you look at some code, can you trust what the function name implies is done, without concern?
When something is complex in a relationship we pause to take time to communicate and come to a common understanding, we write notes to each other.
When we have something complex in code do we write down information to help the other engineers work through it?
In a lot of ways the way we relate to our peers through code is possibly a reflection of how we relate to others in life.
So when reviewing my carelevel is highly dependent on if shared code is touched. If the commit only contains changes in the leafs eg subclass / single microservice my gut instinct is to trust the code and go into LGTM mode (if it goes wrong it's at least localized). But if shared code is touched I don't trust it, I deep dive and complain about everything I can think of.
But I do hate clickbait.
Writing "readable code" is so damm hard.
Readable code leads to maintainable code, which reduces tech debts.
Alright, many engineers just knows how to fix the bug and call it a day. It's a disaster thinking.
I tend to avoid the “let’s rewrite it” engineers - they’re usually going that route due to lack of practice and skill in developing software. There are times for sure a rewrite is necessary, but IMO it’s sort of like blaming the compiler for build errors. Usually it’s not the compiler, it’s you. But rarely there’s a compiler bug and you’re justified in asserting it as such. Likewise, rarely does code need to be rewritten, you are just unskilled at code surgery - so practice. When you have practiced enough you’ll see that the initial revulsion you felt at their code was mostly your brain reacting to the unknown. The people who wrote that code are often as good or better than you, and understood the domain a lot better if you’re new to the code base. Show a bit of respect for those who came before and learn to learn.
Of course 99% of everything is crap, and no one like to ingest crap.
Add to that impossible deadlines and usual exponential accumulation of hot fixes to a point where a bright new product will be more effective than paying the technical debt. Crafting software is on far worse road than most form of art.
All that said, yes, there are great peaces of code that are a delight to contemplate.
put another way, 90% of code is a liability, 10% of code is an asset.
i’m not sure any code ever moved from one group to the other, though good ideas may be stolen sans code.
greenfield is the only way to grow that 10%. it’s the reason startups exist.
it’s like all the failed rewrites. those engineers gained knowledge and fitness through that failure. if their current employer doesn’t retain them, that knowledge and fitness will pay dividends to the next one.
assuming the spent engineering hours would have been wasted elsewhere anyway.
ymmv.
>Don't write new code when you can use, improve or fix what already exists.
Caveat: Refusing to write new code often means pulling in and/or writing a whole bunch of extraneous code.
Senior engineers also hate dependency hell, having to patch other people's code because upstream hasn't fixed it yet, knowing that might break on the next update, yoinking in massive complex frameworks and libraries when you could've just used a few simple functions, and having lots of kludgy plumbing code to wire up all those third party dependencies and generic abstractions.
All those things that people do just because it's best practice not to "reinvent the wheel". All those things that require extra work, maintenance, and system resources, but aren't directly related to solving the business problems. All that time wasted not even working on the valuable domain logic.
And then when you do eventually get around to implementing or modifying a domain case, can you even find where the domain code is in all that mess of extraneous code?
[1] A couple of people here doubt it being true. It is though, and I say that as someone who enjoys reading interesting code. A good case in point is this great, probing interview by Peter Seibel of Hal Abelson:
https://gigamonkeys.com/code-quarterly/2011/hal-abelson/
Abelson: Read a lot of good code. That’s the real issue—that people don’t read code.
[...]
Seibel: I want to dig a little deeper on this. You, like many other people, say programmers should read code. Yet when I ask what code have you read for fun or edification, you—also like many other people—answer that you read students’ code, which is your job, and review code at Google, which is also your job. But it doesn’t sound like you sit down of an evening with a nice printout and read it.
Abelson: Not for a long time.
I can't imagine a professional writer answering like this.
That's a good way to put it. The nuance is that, most of the time, "deprecated" means "not going anywhere anytime soon, so now we have 2+ subsystems for the same thing."
You really do need someone with a grand plan to keep this in check.
You can make so much money selling hoax ”zero code” solutions, as management is very happy to drop money into projects that promise to replace coders with Magic Product(tm)
It would probably be easier if there was a video or scrubber/slider to see how the code evolved.
So many times you look at something and go: why is this so complex. Could be simpler. And you miss an edge case that causes the need for the complexity/abstraction. There are times too though that after a refactor someone can’t see some redundancy or is hanging onto an abstraction that looks great but is unnecessary.
Can definitely relate - PRs with lots of new code immediately trigger alarm bells for me, and while reading through all that new code ain't necessarily fun, coming up with ways to reduce it is a worthwhile and rewarding challenge - unfortunately you're then stuck with the thankless task of convincing the author why they should throw away all their hard work.
code that is simple, documented and well written is best thing any developer can inherit.
I've analyzed myself in process and it's because I get bored and start glossing over details.
In some cases, I feel this makes me a weaker engineer than my detail-oriented counterparts.
In other cases, I connect concepts and problem solve better than they do because I am quite smart.
The division of labor, I guess.
(In this comment: Software engineers also love to sass one another)
They hate when they're put in front of horrible code and they are disempowered to do anything about it.
Or they are forced to agree with people with learned helplessness.
My preferred heuristic is: "will I or any other developer be able to understand this—quickly—in six months (preferably without comments)?" If the answer is "no," refactor using a different approach.
OMG this is brilliant