There is no software maintenance | Hacker News Reader

There is no software maintenance

henrikwarne.com

133 pointsnikbackm88 comments

The idea that software doesn't need maintenance (for any reasonable definition of maintenance) is ludicrous. When a new vulnerability is found, fixing it is maintenance. When the cloud providers change their setup in a way that requires updates, that's maintenance. When user growth causes scaling issues, fixing those is maintenance. When users move to a new platform that needs support, it's maintenance. Claiming that such changes are new features is marketing bullshit. From the end-user's point of view, none of those add new features, they just keep it working the way they're used to.

If there’s one silver lining from the Great Southwest Airlines Meltdown of 2022, it’s that software engineers will have a good and memorable concrete example to point to, when non-technical management or users question whether software/IT process maintenance is “really” necessary.

> The idea that software doesn't need maintenance (for any reasonable definition of maintenance) is ludicrous.

Fortunately this idea is not mentioned anywhere in the article.

Just in the title.

I agree in principle however I have knowledge of software installed on computers in warehouses that haven’t been touched in 15+ years and get daily industrial use. Yes these machines are not internet connected and run ancient operating systems. Obviously web based applications have a comparatively massive maintenance burden however not all software uses that model.

That sounds more like technical debt without a payment plan.

Imagine that all of those physical machines stop working tomorrow -- let's call it the fictitious but notorious y2.023k bug. Imagine the software won't install or run on the a current day computer+operating system. Imagine you cannot install the old version of the operating system on new hardware (missing drivers / the operating system vendor pulls an Apple and explicitly prevents that / it's against the ToS and you'll get sued / your license has expired / etc).

Now what?

What the author calls the product model is just capitulation. There's no agreed-upon requirements and no scope. Each product thus occupies the team in perpetuity. That's obviously bad for business.

I would argue that this model does describe maintenance, but a very bad form (due to the permanent addition of ill-fitting new features to justify the expense). In fact, I think that scrum's failure showed us that projects (what scrum apologists try to disregard as "waterfall") are the way to develop software. Iterations are the way to maintain it. One can put small features into the latter, but everything that's large or has business value deserves a project. You don't know the requirements yet? Then go get them before we start development!

Note that projects don't need to be long and projects can easily hang onto an established agile maintenance process, e.g., for releases and other infra. But they need to be planned properly, have an agreed-upon scope and don't need to fit into arbitrary sprints.

Bad for business?

I think worse for business is not maintained software. When you move devs to another project after 6 months they lose connection with old code. Fixing anything non-trivial becomes 10x more expensive.

When you have team constantly working on code it is a lot easier to have stuff fixed.

IMO that is why we see everything moving to a SaaS model.

Where SaaS model allows multiple companies to get some software running in perpetuity by essentially sharing cost by paying subscription.

No everything needs "team constantly working on code". For example few years ago I wrote a project which takes data from one internal service, parses it, does some processing and puts it into a different internal service. This code has been running 4 times per day ever since. I don't touch it much anymore, except few times per year data formats change, I get a failure alert, and come back to it and fix the parser.

This code is in 100% maintenance. It is done - the existing functionality works reliability, and while I can think of some new features, I have no intention to add them as I am busy with other things which are more interesting and important.

Does it take me longer to fix bugs because I don't work on code daily? Probably. Would it be better (for my happiness or for business) if I start adding features people don't care about to maintain "connection with old code"? Nope.

So what you're saying is that, except for all the times you come back to work on the code because something broke, it works reliably? Nothing about the parser could be improved so that it doesn't break on data format changes? Nothing could be improved such that instead of alerting you to failures, it could be pro-actively adjusted to accept new formats before something fails?

A parser which does not break on data format changes is called "non-validating parser", and I am not a big fan of those. I think I can do better for my downstream consumers.

Let me give a hypothetical example: in report FOO, field "Secondary Record" used to contain record number but now it's something else. Should the parser change output type to string? Or try to see if there is a record number somewhere in that string and return it? Or just pretend that this unparseable field does not exist? In my opinion, all of those options are terrible. The parser should stop parsing and alert human maintainer (me). A human will look at new data and decide what to do with it - maybe change the output type, or maybe there is now a record title as well which needs to be handled separately.

As for "pro-actively adjusted" -- well, that would be great, but the data comes through a long chain of services, and while each individual one has tests, there is no overall test for the whole chain. Could I make one and hook it up to each team's CI, so that parser never fails? Sure, but the service is in maintenance mode, so I am not going to do it.

Right now the downstream consumers are OK with the fact that sometimes data stops updating and they have to live with the stale data until I fix the parser. If they ever want more, great! We can set up the project, decide a priority, allocate time (mine or someone else's) and do it. But until this happens, the project is in maintenance mode.

What if the systems they are building provide more business value than minor improvements to a system that functions 95% of the time in a given year? That work to fix it is maintenance and you might be able to argue that given some timeline there’s a break even for the time investment for an improvement, there’s also opportunity cost associated with it.

There is a ton of software that is less a true product and more an integration of systems that only gets improvements and updates when things break. That’s maintenance and you weigh when to make larger changes against what’s valuable to the business, how much toil is associated with it, impact on team morale, etc. If you don’t you’re just making uninformed decisions for the sake of some fuzzy metric representing how connected to code you are.

It's all about framing--one could say that adding a new format is a new feature and not maintenance just as easily as saying it is maintenance. The problem with calling it maintenance as the author points out, is that people don't understand what that means unless they write software.

I feel like this is a false dichotomy in the real world anyway. At almost every agency I have worked for we have had both types of clients, and a spread inbetween.

Some are on retainers for ad-hoc support, some are under constant feature development as that is what the business demands, and some we put together and handover to the client complete per a spec.

We don't even use scrum/agile on every project, some just have really light weight agile scaffolding, some we do waterfall if it's small and the client doesn't want to participate in the goings on. It isn't one size fits all in software dev, clients have various situations and expectations you have to meet.

From my point of view discussion is about what was written in the article: "project", "product" or a "system". Which for me requires team of developers, QA, business analysts and cannot be grasped by a single developer.

I also have small tools that can run after 5 years but I don't see these as interesting argument in the context of article.

> Fixing anything non-trivial becomes 10x more expensive.

It’s the same model as car repairs. You can do checkups and small maintenance every month, or have the whole thing checked and repaired when enough time passed or something critical broke.

IMO it comes down to wether you can afford for it to break . If it’s a subsystem that can be broken for 2 weeks, regular maintenance might not make much sense for instance.

Yes it probably can be compared to car repairs. But often the programmers will promise that it will be much better to introduce the new hunken dakser reactivity framework. And this will then continue in the infinite scrum project.

IMO, you need multiple models. The product model (or the "this team owns X" model) force a commitment to an area of work that doesn't always align with business needs. The project model is terrific for aligning people across skill sets, but doesn't do as well for ongoing work and tends to push staff from project to project (vs building deep experience), which can work well for generalists but not for everyone.

I see the actual problem being that management is rarely simultaneously deep enough and strong enough to make the right choices around work organization so everything just defaults into hierarchies.

To note, "this team owns X" still has the team justify a budget or even its existence when push comes to shove.

For instance if there was a “blue check” team of 3 people at Twitter, they’d have a hard time justify keeping the team 100% committed to it for years after the feature is released even if there is some small amount of ongoing work needed.

Looking at estimated output vs the costs, they’d probably hibernate the team most of the time, as you do with projects basically.

If your core business is software, you should always have people working on software. You don't stop improving things once they are built because that's how you fall behind competitors, whether improving means maintenance or new features or a mix of both depends on having a good PO.

If your core business isn't the software you're building, I tend to agree, but then building instead of "buying" would usually be a mistake.

The core business is rarely the software. We're not all lucky to work at Jetbrain or Atlassian, some of us work at banks, search engines, social networks, marketing web agencies and stuff like that.

And buying is impossible in a lot of cases: if we're gonna be differienciating vs the competition, we wont use an off the shelves software, and sometimes the clients want customization so complex, we could have bought the thing and still need a big team to customize it.

I think the guy is basically saying projects need to come back, which I can accept but then we need to know: are deadlines good or bad ? is the client supposed to invest his time in the product or wait for completion ? can requirement be known before progressive iterations have shown a building skelleton to the client ? are developers wasting more time waiting for project design completion or fitting arbitrary scrum sprint scoping ?

> You don't know the requirements yet? Then go get them before we start development!

Couldn't really disagree with this more.

Sure, don't start development before you know any requirements. But also don't wait to start development until you (think you) know all of the requirements.

Why not?

Often times customers don’t know what they want until you start to build it.

For the same reason that any other exercise in predicting and future is foolish.

You can plan all you want for the first release, but as soon as users start using it there will be changes. If you want to make that a new project with planning that is up to you, but that fact will always continue. Release, change, users demand new changes etc. And that is so true. I've never had a project that was truly finished, also because i suggest new features, especially when I see how they use it.

The client doesn't know what is possible, we need to understand their business.

There are projects (defined end), and there are programs (indefinite). Sounds like most computer programs are, oddly enough, the latter.

I think it's worth mentioning the Cynefin model here [1]. Both approaches can work, getting requirements up front can be the right approach for a well-know domain, but often times it's better to treat requirements analysis as a discovery that is ongoing during development.

For the latter case, classic project management simply leads to failure and it needs iterative development with tight feedback loops.

[1] https://en.wikipedia.org/wiki/Cynefin_framework

> In fact, I think that scrum's failure showed us that projects (what scrum apologists try to disregard as "waterfall") are the way to develop software.

Ye I have come to this conclusion too. I am not even reading "waterfall" as a pejorative anymore after all the agile I have had to endure.

"Every time I hear about software maintenance as a distinct activity, I cringe."

The author seem to suggest that never ending agile grind is the way to go and that programs can't be more or less done. Maybe he is doing Cloud™ stuff SaaS, where ongoing costs are a feuture not a bug.

There's a reasonable point in here, that for some companies, there is no distinction between maintenance and ongoing development. You're just constantly thinking of new features, and the work of building those features and changing the way existing features work is comingled. In that case, you have no such thing as maintenance. This model is probably more common, thanks to the rise of SaaS, and automatically updating end-user software.

But sometimes you sell a product, and the product or some component of it (perhaps some integration with another tool) of it really is in maintenance mode.

You decide that acquiring new customers is unlikely, or not worth the cost, and you make the bare minimum of changes necessary to keep the lights on. No one touches it for months, and when they do, there's a bug fix. Perhaps customers have new ideas for using the software, but you're not going to do them.

That's maintenance mode. It still exists. It's not a fun place to be, but it's real. It even can be valuable (since fixing the occasional bug report can be very low effort, but necessary to keep customers around).

A lot of software these days has api dependencies for the things it works with, so it stops working if you stop working on it.

Not only API but world is moving and any non trivial app seems like would need to be updated at least once a year or twice. Because waiting too long might make future updates harder or even not possible.

One of my biggest complaints about depending on _anything_ not in the language's standard library, even standard extensions might be deprecated.

Even standard functions can be deprecated and removed over time. A prime example of that is C's gets() function.

That's what maintenance means - you are going to fix regressions (reduction in existing functionality) but not add the features.

A lot of established APIs are pretty stable, so you may not even need a single full-time person (when averaged over a year).

Maintenance can be interpreted in different ways. In my software career it basically means "no new features" and "bug fixes for a known duration". It's more a project lifecycle status to be understood by customers/stakeholders rather than a distinction in software development activities.

I hope not sound too critical, but the authors product model perspective is one I would expect a junior software developer to propose; as it does not consider a wider perspective that includes sales/business side of things. When I sell a software product, customers have certain contractual expectations about what they are buying, and what support they are paying for.

A straightforward "product" is a fairly comprehensive solution that does what it says on the tin, and gives enough people their money's worth to be viable as is without an actual necessity for any ongoing support or maintenance in the simplest of cases.

Such a product can well be developed on a project basis and with successful completion the development can then halt altogether and those resources redeployed in a subsequent project which can actually be completely unrelated to the first.

But the simplest of cases is not what is usually found.

At the other end of the spectrum the hardware area has some good examples where sometimes the physical hardware "product" purchased and delivered amounts to a much less significant contribution to the Total Cost of Ownership when it comes to the product aquisition itself. Some products just by nature or design whether because of complexity, or maybe just defects, aren't really that useful without essential ongoing factory support and maintenance which ends up costing much more than the purchase price of the "product" initially. Sometimes even a hardware purchase is just the ticket in the door for the manufacturer to provide more profitable follow-up services, resulting in more steady cash flow for years to come.

Things can get a bit nebulous when the product doesn't really have a tin to say everything you need to know before buying it, but it's definitely possible and sometimes the only acceptable thing, for the most suitable software product to be the one that requires no ongoing support or maintenance.

Unlike hardware there's nothing physical to wear out or replace. So there'e even more chance with software to come up with a clean separation between those projects which are best brought to successful completion, resulting in a product that can stand on its own, versus having the project itself be the actual "product" and having that be never-ending.

What the author's saying is everything should be never-ending which is ruling out some of the most effective solutions to some of the most important probems.

This is an unhelpful oversimplification. There are lots of tasks that are maintenance, and the more code your team has written the more these tasks build up. I don't know what motivates someone to pretend it is the same as writing new features, but it deals with distinctly different risks.

I think it's sensible to be critical of the language used in our craft, especially when borrowed from the more physical world, because with it comes the baggage. For example, I have a pet peeve with the phrase "use the right tool for the job" used in the context of languages/stacks, because languages and stacks are more akin to building materials than tools (IMO). Selecting a building material and selecting a tool involve different considerations, and thought-terminating cliches like the above can give a false sense of clarity. So IMO, to look at software maintenance as the author has done, I think is fair and reasonable, even if just to render obvious that software maintenance is a special variety of maintenance that differs from most usages of 'maintenance'.

I would just say that when the effort of fixing bugs, refactoring, re-platforming, updating docs, and tweaking features, is greater than the effort invested in major changes, you're in maintenance.

Most software applications get there. Take your favorite IDE. It's in maintenance. Vim and Emacs are in maintenance. Your bank's web app is in maintenance. Chrome is in maintenance.

I admit this is somewhat arbitrary, but there are things that you do less of, and things you do more of on a lot (most) of production applications with users. Sometimes the needed functionality target was missed the first time. Maybe the second time. Maybe users massively changed their minds. So you might not get into maintenance for a long time, but that might be more due to a poor understanding of what the market wanted, rather than there being no such thing as maintenance.

Absolutely good points!

I used to work in a (Scrum) team where 80% of our projects were in maintenance mode exactly the way you described. It felt horrible to work there as an engineer (at least for me who enjoys creative processes and development).

You still get all the meetings, administration, rituals, debates over minuscule stuff - without delivering any customer value. There were entire sprints where most of the team hasn't shipped any useful feature, only fixing cryptic bugs in "other people's code" that nobody really cares about or keeping up with the latest changes in the NIH syndrome-ridden microservice environment.

Realizing that such a large organization can't be fixed, I moved over to a different company where the engineering team works more with a "product engineer" mindset. Probably the best decision in my career so far!

We're still "maintaining" software, fix bugs and refactor etc., but due to how Shape Up [1] works, creating customer value vs. maintenance is almost always 6 weeks vs. 2 weeks so 3:1.

But even for a Scrum team in a large org where some projects are passed around like hot potatoes, I'd never assign more than 50% maintenance mode projects to any team. This may not be the most efficient from the organization's standpoint, but it would probably do wonders for preventing burnout and employee churn.

[1] https://basecamp.com/shapeup/

> Chrome is in maintenance.

Unfortunately this is not good example, because Chrome undergoes continuous refactoring and is always adding new web features. Even V8, which I worked on for almost 7 years, is definitely not in "maintanence", as JS changed radically in that time and we added the Wasm engine subcomponents.

I kind of agree with your other points, but this is really a spectrum. For example, a 115 year old house is in "maintenance mode", yet it needs new insulation, wiring, upgrading the heating/cooling, interior redesign, floors, windows, etc.

The truth is that most long-living systems are composed of many different subsystems which run the spectrum from "install once, fix it when it breaks" to constantly being rejiggered.

But I would say, continuous refactoring is maintenance. You would not need to do it unless there was technical debt that had not been addressed in a timely manner. And when you are forced to do development because your underlying platform, i.e. javascript, changes, that's maintenance. You're not adding anything, you're just adapting. Though I concede that to the extent that the JS changes allow new capabilities and you engineer them in, that's a gray area.

You can think of maintenance as one sub-branch of software development, usually containing tasks that do not create immediate value for the business, but need to be done to be able to add new features. It's also a good way to make it clear to non-programmers that a certain amount of time needs to be spent on tasks that seemingly do not create direct value. Maintenance is a concept that everyone understands and is a good way to explain it.

Regarding the project vs product model, it fully depends on the business. Venture outside of the SaaS/FAANG/etc type businesses that HN seems to focus on and you'll see countless businesses that are very happy to just have a project done for X cost in Y time. After that, the project goes onto a maintenance contract, where the development team is just expected to keep the lights on, fix bugs and keep it secure. From my experience, it's a huge part of software development, considering the amount of systems that exist in this form is only increasing.

This is exactly what I was thinking after reading the article.

Projects develop new products but are expensive to undertake. The enterprise products that resukt from these projects are largely profitable though deployment and maintenance efforts.

The challenge here is that while most businesses assume software is just "produced" once and then earns them money indefinitely, the reality is that there is some amount of "production" that needs to happen afterwards. Most dichotomies between ops and dev also come from the same gap, which is fundamentally not a "skills gap" - it is more of a "software developers are expensive, can we have 1 less expensive ops person supporting 10 services than the original 10 developers tied down in maintenance?". Businesses which are smart and understand software well know that regardless of how well you play it, there will be ongoing development after the initial release is done. Modern software which runs evergreen and gets worked on continuously is even less menable to the idea of "maintenance".

The issue is creating a shared understanding that "software is never done-done". The decaying corpses of marketing iPad apps from 2010 show the only area where this is not the case - there is one very specific period when the app is relevant, and everyone knows that after nobody cares about the software even if some users would.

What can help against the "maintenance trap" if the business is listening:

* "Legacy" is a taboo word. Replace all instances of "legacy" with "this pays our salaries" * When you build a system, stay cognizant of when you want to put it to rest. If the system outlives that horizon - which often does happen esp. if it does pay your salaries - examine and extend that horizon. * Be very, very alert about people let into the organisation who pitch a "full rework to get rid of all the maintenance", and - if the system already in place is not absolute shite - just give those people a pass.

The topic is very, very underresearched and the only article I saw about it which was truly astounding was https://apenwarr.ca/log/20171213

Software maintenance is what you have to do for it to keep working. It is a useful distinction, especially from the point of view of management, even if the tools and skills used are the same. In many companies, the funds for maintenance might come from a different source than developing new features.

One could say that all office workers do the same job -- reading text on the screen, clicking with their mouses and pushing buttons on their keyboards. But this misses the reasons why people are pushing those buttons or clicking things with their mouses.

Exactly. Maintenance is keeping the lights on. As a manager, it is the level of budget you can't do without.

They sorta handle this in the end, which I thought was thoughtful, since it does contradict their original point a bit.

> However, if you consider that the environment the program works in can change (library or OS upgrades for example), then you could compare this to handling wear and tear.

> There are systems that are maintained only in this sense: fixing bugs, and making sure that it can keep running. But I would argue that this is a very small part of all software development work being done. Furthermore, when it comes to fixing bugs, there can be ambiguity. Is this really a bug, or is it in fact a request for new functionality? And why fix the bug at all, if it has worked up until now, and the only objective is to keep the system running. So, in some sense, this form of maintenance is also just ordinary software development.

I’m not sure what to think about this in particular. A program runs on a computer, which has an OS, and over time those OS’s are updated. Is “make sure it runs on RHEL 9, when it already ran on RHEL 8” a development or maintenance task? I would characterize maintenance as dealing with the inevitable decay of things over time. Technically the computer+RHEL 8+program system should still work but of course you probably want to benefit from the new development work put into RHEL 9. And if you let your OS become too old, it becomes unsupported and insecure…

Where I work, the difference is basically, making things that already bring in revenue continue to work = maintenance. It is about the decay over time, but also the cost of just making business, keeping the lights on. Just as you need to pay your cloud bills to keep your service running, you need to have a certain amount of developer-hours addressing things that do not work perfectly and needs adjustments and fixing. For the developer in question it may not matter much, it is git branches and continuous deployments just the same, but the distinction is not really for the benefit of the developer. It is for the manager that can reason about different opportunity costs by knowing how much time and money maintenance work needs.

Another perspective not mentioned is that there is a real context for why the distinction matters in the US. In the US, new features can be described as capital expenditures, or CapEx, whereas maintaining the fitness of the software is considered OpEx, or operational expenditures, and both are taxed differently. This matters for public companies.

This matters for R&D tax credits as well. Only new feature development is considered eligible.

Private companies too. And the P&L looks really different depending on whether you expense or capitalize everything

Depends on the domain. There are millions of native Android and iOS apps that are only (slightly) updated when Google/Apple forces developers to.

This matches my experience with software development.

I work on business software that serves an industry for at least 10 or 20 years, and there is never any end to development. There is always value in making the industry-serving software better at serving that industry.

Sometimes that means fixing bugs. Often it means tweaking the schema to better match real life.

There is maintenance! Build any corporate software or mobile app, things will stop working eventually due to api integrations or deprecated dependencies. Even if you leave your software untouched

When you talk about software maintenance you're trying to get a hold on the cost of software. Most projects have a full development cycle to 1.0 and that after the initial release that team may be prioritized to work on something else (or not). When this happens the original piece of software may be in production and still needs to be maintained but no active development is taking place. The reason you might do this is that the dev team is a precious resource and having them iterate forever on the same piece of software (the product model) may not be financially feasible or desirable to the company.

There are companies that can afford the "product model" but it would require that everything developed by a company maintains a full dev team per software component. However, not everything needs to have features being added to it continually. It makes more sense from a people management perspective to move that team off and have them work on something else. In which case you have something in maintenance.

If you have to budget software it does make sense to think of it in terms of the capital expense of upfront development and the operating expense of it's maintenance. Much of the operating expense may involve hardware upkeep, recurring licensees, etc. People are an operating expense too but they can be moved around. Nevertheless the 'software' (without you) needs to be thought of as something that will be maintained when you're not working on it and what the cost of doing this will be.

I don't think I agree. There are two distinct activities - developing new functionality, and keep existing functionality going. In an IT organization, you may for example get an incident in production, that requires fixing and deploy of a new version. That is maintenance.

I would argue that this too is part of software development. You ought to have a robust process that can accommodate hotfixes while minimally impacting ongoing work. That fix requires you to get buy-in from all of the right stakeholders, and the deployment requires you to follow the same processes as the development team.

I'm not sure if I see the value in making such distinctions though.

Yes it is part of software development in a larger sense. But there is value in it if you work as a manager. When you create your budgets you need to have an idea about how this work is split proportionally, because the work affects revenue in different ways. From a budget point of view, you can see it like: there may not be enough money to fund new feature development and the effect of that would be unfortunate. But if there also is not enough money to fund maintenance, it is worse.

As a freelancer I disagree. I get hired by clients, do a product/project, they use that product in production and/or deploy it to their users. I get paid for that phase and then either we enter a 2nd phase of maintenance or the client goes with somebody else in the future for the project needs. When I do enter the 2nd phase, which can last years, usually my work on that project is in average to 10 hours/month vs. the full 40 hours/week when was the initial development phase.

So yeah, there is a development and there is a maintenance phase. On big projects that can spawn years eventually the maintenance phase can be seen as continuous integration, which is what the article's author is talking about, and that phase can be 90% of the project budget/lines of code/lifetime but is still the maintenance phase.

As Ray says it, he is seeing this through corporation colored glasses.

What this completely overlooks is the time developers spend trying to understand existing code. For greenfield development that is close to zero. For maintenance work it is over half of their time and arguably the most difficult work in software engineering. Maintenance work in complex systems is far more challenging than developing new complex systems.

Some people, not many, take software maintenance seriously and conduct studies on it - in the study at the link below they point out that comprehending existing code took on average 57% of developer's time. Actual editing consumed 5%.

And software maintenance is no different - think again.

https://www.researchgate.net/publication/318811113_Measuring...

I've worked in place where the majority of projects and the majority of code went in to maintenance. There could be several months or even a year or two of intensive development, with teams reaching a dozen or more developers + QA, DBA, PMs, Ops and a bunch of people on the client side.

And then those projects passed UAT, then went on production, a round of change requests and the teams were disbanded, with maybe a single developer giving some of their time to fix a bug. Sometimes such project went many months or even a few years with no development time spent on them. Some had backlogs of bugs, but it was very hard to mobilize resources to fix them. The company was chasing new project that were bringing more money, so after delivery the old projects had huge opportunity costs, even if there were maintenance fees.

You can play with semantics of course. But the fact is that most software development is not building new software but doing development on existing software. That has traditionally been called maintenance.

There's a clear difference between the early life of a software product where there is a lot of fast paced development to create the product, deadlines, etc. and the maintenance phase of a product where most of the team goes off and does new things and a smaller team remains to keep the thing going. It's a bit of a grey area where one stops and the other begins. And of course some popular products keep on evolving at the same pace because there's a large team working on them doing major new releases once in a while.

However, there's a lot of software out there that is still being tinkered with but that does not really change that much over time anymore. Most of that development is just keeping it going, doing some small changes as needed, adding features, fixing compatibility issues with new hardware/software, etc. This usually gets broken down into perfective, corrective, and adaptive maintenance. Software is never really done. If it's useful/valuable, people will keep on coming up with ideas on how to make it more useful.

What sets maintenance apart from regular development is that it is more reactive and unplanned. It involves less people and might not even be a full time job for the few people that are still able to work on the system, which usually requires having some history with the system.

This has been my experience. I have been working on an in-house built LMS for a Fortune 100 company for 20+ years. People keep wanting to make changes, add functionality, redesign the user interface, etc. And all of these things tend to introduce bugs. There is no end from what I can see and I wouldn’t be surprised if I worked on it until I retire.

Software maintenance has multiple distinct forms:

1. Perfect software that doesn't run in a vacuum. Assume the software has no "bugs". It still operates within limits, and within an imperfect world. The hardware fails, it gets upgraded and is no longer compatible with the original software, the disks fill up, invisible particles shooting through the universe flip bits. All sorts of things outside the control of the application will cause the application to not function as intended. And as such, maintenance will be needed.

2. Imperfect software run by imperfect people. Even without any of the above problems, users will input the wrong data that the program didn't expect and cause an error. Sure, you could just assume that "software development never ends" (assuming that you will just find new bugs constantly and need to fix them constantly). But you don't have to fix a bug if an acceptable mitigation exists. People who still run Windows 98 or XP can't fix bugs, but they do work around them. And so maintenance can take the form of performing tasks which aren't changing the software, but are tasks that make it continue to function.

3. Actually shipping updates to the code to fix bugs. This isn't strictly necessary. There are many bug-riddled software systems that have been going for 40 years without patches (for the most part). But in the modern era, we have so gotten used to the website model of daily updating a piece of software, that this is what we assume 'software maintenance' means.

Once you write a piece of code, you might continue to maintain it using #3, but even if you don't, you will absolutely have to perform #1 and #2, if you want it to keep working. If you are very lucky, and the system is "on" but doesn't actually do anything, you can get away with not doing any maintenance. But the more you use it, the faster it needs maintenance.

I've often told people "There is only one program: Hello, world! Everything afterwards is maintenance."

The moment a piece of software is released and in users hands its in maintenance, which is the fixing of bugs and production issues. But that does not mean its also in development, the process of developing new features is quite often a distinct activity and not something that has to continue.

I guess this is an interesting thought experiment but by the conclusion I felt like it's just splitting hairs on terminology.

I don't really agree that the line between a bug fix and a feature request is blurry. Those two activities are clearly distinct in my mind.

Bugs of logic omission could easily be argued as feature requests. Who is to say that a very poorly written app,done cheaply, whether it intentionally omitted proper data validation, or whether adding data validation to resolve downstream errors is a feature request.

Another example is the classic, "users would never expect it to work like this" vs "it was designed to work that way (even though it is ass backwards)". Is fixing something to fit user expectations a bug fix, or a feature request? It can really be that the question hinges on an original design spec, which seemingly should not be determining factor (but it is, and who knows whether it was just done badly, or done wrong)

Ultimately all that matters is how many person-hours an organization dedicates to a product that’s considered low priority, finished, deprecated, whatever you want to call it.

Debating all the linguistic similes we can use to describe the process is fruitless.

Despite that freedom of interpretation, I think your hypotheticals have really clear answers that most software engineers would agree with based on what was originally promised and intended.

If a user expects a sorting function to have ascending and descending, but I only originally delivered and promised ascending, later adding descending is a feature/enhancement. It doesn’t matter that the user didn’t like my original implementation, I didn’t promise descending.

Conversely, if I add the descending function because I originally intended for that function to be there and wrote some amount of code for that function, but it doesn’t work, that’s a bug fix.

I don’t understand the ambiguity, the difference is clear as day to me.

In any event, the whole debate revolves around the language of intent. Truthfully, a group of people can make up whatever language they want to describe their approach.

Unsaid, part of my point is the bug vs feature request is a bit of a shell game. Absent the developer & the source code, all that the manager might know is that they are getting support tickets for "this is not working as users expect, we are getting calls into the support center about this." At that point the manager can then decide to do the shell game of "well, we have this JIRA that says it works like that", or, they may choose to say "we are getting bug reports about this, the product does not work as the user expects", and hence it is a bug item. Which then gets into the whole politics of backlog prioritization (feature requests often go to the back, AKA the trash-can, and bug reports maybe get more love..)

I think this kinda goes to the point of the article. No matter whether you call it defect remediation, bug fixing, feature work, it's all the same thing - updating an existing software system. I think the point is that the very _vast_ majority of work in software is on existing systems and its almost all is more-or-less the same thing. Which is to say, "maintenance" and "enhancement" are just about the same activity for the software developer. EG: if a feature can be added with a quick 'if' statement, and a bug fix can be fixed the same, are they really different activities? What are the implications relative to the time to understand the system before making that modification?

Essentially my answer to your question in your second to last sentence is “yes” when we zoom out to the business level.

It doesn’t matter that the physical activity taking place is the same, because what the activity is for determines where the company puts the expense on the income statement.

That's the point of the article, no? For a business, they can choose if they want to define a bug as "does not work to spec", or "does not work to user expectation" (I believe we agree there, that this is a distinction without a meaningful difference to development)

So, for the developer, the activity is the same. Given most software is existing and not green-field, if you extrapolate this idea out further, it implies that feature and maintenance work have a lot more overlap and are fundamentally the same thing for even larger projects /features than just small bug fixes. It raises the question, at what points and when does feature work and maintenance work become distinct activities?

I have been working (I avoid using either "developing" or "maintaining" here on purpose, tough I would have used either of them in another context) on the same code base (mostly) for more than 10 years (doing code, infra and operation)

Well, we can nitpick on the wording all we want, but there are times when I feel I'm cleaning my apartment or repairing it (maintenance) and times when I'm extending it (developing).

Now if the point of the author (I'm not sure) is that you cannot develop without maintaining because both go hand in hand then I agree.

The project model is forced on large parts of the industry because no one but the direct customer is willing to fund the development of a bespoke piece of software.

This brings about the distincion of the phases pre and post acceptance.

The 'maitainance' is a customary upkeep as no one but the outside developer can in practice economicaly assure the system's ongoing operation. It also incentivises the service provider to respond to keep servicing smaller adaptation requests.

Fun thought: this looks like a problem that could be solved with insurance. The customer pays a flat maintenance fee for as long as they want. The developer pays damages for any software faults and insurance premiums that guarantee that the customer will get their money back if the developer goes bankrupt.

Similar to malpractice insurance in other industries I believe.

It is mostly not about late discovered faults, but adapting the software to changing environments outside of the product's code. The customer on the other hand is mostly very less interested in getting their money back, but in a functioning system. Code in most project agreements is already in the hands of the customer, or at the very least placed in escrow.

And when you say 'the developer pays', that is 'the customer pays', as the insurance premium would be embedded in the project cost. Apart from the insurance industry, who wins here?

The customer wins by being able to compare prices. The developer can no longer make lowball offers and expect the locked in customer to write blank checks when poorly written code blows up.

Sure, but this is sort of like saying there is only one programming language because they're all turing complete. Code produces behavior, and that's the actual goal, so why do we bother separating them and talking about it? Produce behavior! Wait, no, we're all just workers because farming is also just behavior. If you really think about it, it's all just change, let's refer to it as such, not "making a bridge" or "toasting bread".

For there not being two distinct phases: well.... it depends on how you draw the line. "Prototype works" -> every change after that is arguably maintenance, not development, since it has been developed. What you're doing is then "just" handling its interactions with the real world - roughly equivalent to repair (fix bug / replace broken component) and maintenance (prevent total failure when one host dies / lubricate parts so they don't fail as quickly).

We don't refer to lubricating and replacing parts as building the machine though. I'm not building my laptop when I plug it into a charger.

---

Even if we both stop taking things to absurd extremes: I agree things are not "developed" and then "maintained" in most modern, always-updating systems, but I can definitely separate much of what I do into "development" and "maintenance". Development is whatever is focused on getting a thing working. Maintenance is whatever is focused on adjusting the system to make that development reasonable beyond the minimum hack, or make the changes long-term palatable, or updating libraries, or enhancing observability to resolve issues faster, or restructuring things to make future changes easier, or...

The list of things we do beyond initial "development" of a feature almost totally regardless of where that line is drawn is vast, and you can often adjust how much of it you do now vs later... but you generally cannot do "some" development later. It either works or it doesn't. You have a button that does nothing or a button that does something. Sure, you might be able to split a feature into MVP and nice-to-have sub-features, but you can't say "oh we'll figure out the conditions of that if statement next month, ignore it for now".

Softwares state is depending on how close it is to the actual/perceived economic hot-loop in the product.

Thats where features grow, were software is properly maintained and tested. The further away it is from this route, the more it becomes a stagnant resource backwater, until the software - which might be alive in other sections, actually starts to die and bitrot, basically a forked away stand alone thing, with its own library, tucked into a corner, visited only on rare occasions.

This reeks of the very common "feature factory" (https://medium.com/@johnpcutler/12-signs-youre-working-in-a-...) dev shop. With this mindset you will most definitely have a bug-ridden hard to maintain app in a few years.

I develop embedded software. When products mature and are released they might need maintenance in the sense that the original SW team is not working at the company any more or working on something else.

One or more developers gets tasked with maintenance of this product and can bugfix fast instead of trying to ressurect the original software team.

So in my opinion maintenance is real and often used in different industries. :)

Even if the product doesn't change (much), for embedded (especially for long life-cycles) part of the maintenance is keeping the build working in the environment of the current decade.

Seems like an argument of semantics. I've had many projects with a distinct development phase (high input), and maintenance phase (monitoring, low input, a feature every now and then). To think there is only one gear of dev velocity to me seems silly, as does debating the phase labels

I have had a really frustrating relationship with software maintenance in all of the companies I've worked for.

I've always been a strong, strong believer in an analogy like: How much time would you estimate that nuclear engineers spend operating and maintaining existing reactors, versus building new ones? How about public highway departments and building new roads, versus fixing or improving existing ones?

No analogy is ever perfect, but the only logical conclusion I can reach is that software engineering is a really weird discipline relative to practically all of its "we build things that last years" peers. Its weird because, well, our organizations in my experience chronically under-invest in maintenance then act flustered when new developments don't meet deadlines or take forever; but its also weird because all of my experience screams at me "I'm not even sure if we know how to maintain systems well".

I've seen this in one org (a public company you'd recognize); where years ago we had and met lofty goals like "99.999% availability"; today its a good month if we hit three-nines, API wide. In team meetings with really senior, smart engineers when we ask How Can We Improve This, you start observing this learned helplessness where they suggest small changes like, you know, adding a cache for some external API, or replacing the built-in HTTP client with some other one that has a more comprehensive knob for timeouts, or whatever. I talk with them privately and say: Is this really the best we can do? It isn't; everyone knows that; but we will literally never have the product dev time to really fix the problem; maybe, taking what we've learned from some old system and building a new one, cut out layers of abstraction, reduce and simplify. So instead, we supplement the problem with complexity; and the thing we won't admit is that this is like putting a bandaid on a skin tumor; it may improve the metrics on the short-term, but ultimately it is more complexity, it is more code, and in a lot of cases we've replaced one way that something fails, predictably, with N ways it can now fail, unpredictably, hopefully less often.

The other thing that no one will admit is: If the state-of-art way to fix something that needs fixed is to "take what we've learned from the old thing and rebuild it" (oftentimes, this is the best way; but that's another discussion): No one lives forever, and No one stays at a company forever. Knowledge is Temporary. Through leaking knowledge or increasing scope, taking on maintenance today is almost always easier than taking it on tomorrow; and it usually leads to productivity boons which magnify over time.

I word a lot of that from the perspective of developer experience/productivity, but it fits just as well with Security maintenance. I'm working alongside a company impacted by the recent LastPass breach. They were storing hundreds of internally minted JWTs in there like API keys; no expiration, and no way to revoke. "We have to rotate these"; well, lets take a look at this Jira ticket from four years ago where one of your senior engineers who left three years ago said we need to make these things expire, or at least add some kind of revocation list (ok, never word it like that to people; but that's what we're all thinking). We don't have that; we can build it, but now the app has 5000x the traffic and complexity, so it'll take time.

I share this talk by Johnathan Blow [1] all the time, because its fantastic and in this case he hits on a really good point: We only know how to do things by doing them. If we, as an organization, invest 10% of our time doing everything we can to keep existing systems running and productive; that may not be enough time to even develop the skills to know how to keep them running well and highly productive if we could fight for more time. One of the phrases I hear all the time, across many orgs: "small steps toward improvement". I think people who say that have either given up (learned helplessness), and/or they're critically unobservant of the inherent entropy of software systems. There is, no doubt in my mind, a "rate of decay" of every software system; its different for all of them, its influenced by factors like product velocity, external integrations, language, etc; but they all have it. And if you aren't investing more time in maintenance than your system's rate of decay, you're going to cause your People, your Customers, and if you're very unlucky (cough LastPass) your Business a lot of pain.

But; my special variety of learned helplessness is that I think the people in charge of most orgs actually like pain, and like inflicting it on others. Most likely something to do with the observation that psychopaths and sociopaths tend to rise through the ranks a businesses faster than those with empathy.

[1] https://www.youtube.com/watch?v=ZSRHeXYDLko

Agreed. The author says some intelligent things about why software has gotten so low-quality, and the experience of developing software and using software has gotten so bad, but they think it is a good thing.

Imagine interviewing for a job and the CTO says, "Here at HyperAgileCorp, we believe that software maintenance does not exist. We realized that software development is just the act of adding features to products, and, amazingly, there's no end to the features we can add. Using continuous integration and deployment, we are able to add and ship features fast enough that no part of the code is ever in a stable state. Because nothing is ever finished, there is nothing to maintain.

"You might think this would lead to bugs, but actually, bugs are an impossibility. To have a bug would require a deviation from the designed or expected behavior; a bug is a problem with the code. We don't do any design or have any expectations at the level of the fine-grained behavior of the software. Remember, this is a product, and products have features. Like, the other day, I wanted someone to add tables to our rich text editor. You should be able to click the little icon of a table and it should insert a table. That's a feature. Making the software crash less often could be considered a feature--that is, something we could implement. But we don't know if a missing feature is a "problem" until we evaluate how it impacts the customer and our bottom line. If adding a row to a table crashes the app 1% of the time, or 10% of the time, for example... true, that is an opportunity for a feature (not crashing) which could enhance some customer's experience of using the software, and maybe it could even lead to more sales and a more successful product. Keep in mind, though, that we have a lot of other features we are working on that will also enhance the customer's experience, and we have some ideas about how they could lead to a lot more sales.

"Besides, we ship new versions of our software every day. Even if something 'works correctly' or 'doesn't crash' today, a junior dev might change the code tomorrow and make it work 'incorrectly' again, according to somebody. And that's a good thing, because that change is probably part of some way we are making the product even better. We're not trying to keep this code the same. That's the old 'maintenance' thinking. 'Maintaining' something means 'preserving' it. Preservation is for jams and jellies. We're developers. Builders. That junior dev is probably building a big and important feature. It's not practical for him to understand all the code he's changing. His lack of experience doesn't matter, though, because, as I've explained, his code won't have any bugs and won't require any future maintenance! We'll just keep building."

Reply on news.ycombinator.com