Hacker News Reader

Comment by temporallobe | Hacker News Reader

IMO, low code quality and technical debt is one consequence of the Agile movement. Mantras like “Working software over documentation”, the constant push to shove demos in front of customers, and the battle for metrics, the idea that everyone needs to be “full-stack”, etc. have real consequences.

Building software is analogous to building a house. Sure you can make it look good and deliver that to your customer, but poor build quality will eventually be exposed during your first hurricane or earthquake.

View on HN

I'd say it's more a consequence of the need to put boundaries on the costs associated with developing the software. I've been in an internal development group where costs were not defined. We developed what the company needed internally, regardless of whether it was "worth it." If the budgeting people knew that the silly app created to ease their yearly budgeting process likely cost 30-40k of developer time, it wouldn't have been built since it likely saves 1-2k per year of their time. But quality was good because we didn't need to build it in a week at a cost of 2k.

I'm not saying that's a good thing. But my point is that quantifying those costs for external customers means that corners are cut and speed is essential. _That's_ why the code is low quality.

I don't believe that "true" Agile is the reason for it. Referring back to the Agile Manifesto has allowed us to focus on what's important. Other Agile (Scrum,etc.) in the workplace is no different from waterfall or any other method since it is ultimately applied by those focused on the money and then fails for the same economic reasons.

I would agree with that in general. It's not the only reason I would say. Quantifying can be good. In an internal non-quantified project you still have constraints on the number of people you have around to build stuff, so there are still timelines to meet, because you can't build for the other department's needs if your people are still building that first piece of software. I'd go a bit further and analyze your parent's "reasons" in that light:

    Mantras like “Working software over documentation”

Fluffy documentation that has nothing to do with the actual software as built (aka "out of date" or "waterfall spec that took 9 months to create and that nobody updated when it turned out we had to build it differently in the remaining 3 months of project time" etc.) is bad. And that's what this "mantra" is about. Whether it's an internal or external and quantified project, you had 12 months to build it.

    the constant push to shove demos in front of customers

It's a good idea to demo to customers, if you can, because your customers can tell you if you're on the right track while you are still building, instead of telling you that you built the wrong wooden staircase after 12 months only. They wanted a metal ladder. Whether that's internal or external doesn't matter. It's always a good idea and I would argue that it's easier with internal customers. I.e. very easy to go over to Joe that works for the department you're building this piece of software for. You met him at the x-mas party last year and from time to time you eat lunch together anyway.

    the battle for metrics

Hard agree. Pure reliance on (usually badly chosen, incomplete set of) metrics is absolutely bad because there will always be some high enough people with not enough understanding of everything that will rely on those metrics and those metrics alone to make (bad) decisions.

    the idea that everyone needs to be “full-stack”

Not everyone needs to be full stack. But I do believe that you have either be full stack enough to understand the system from top to bottom or be really good at communicating w/ the other people in the stack to figure out how to build the system properly.

    Building software is analogous to building a house. Sure you can make it look good and deliver that to your customer, but poor build quality will eventually be exposed during your first hurricane or earthquake.

If we want to stay in this analogy, there's one thing that can help a lot. And that is to actually start building in vertical slices instead of horizontally.

A house is not build horizontally. It's built vertically. You build an entire "feature" from the bottom to the top. First feature of a house is the structure. You build the walls for the basement, put in a floor and build the first floor, then same for the second floor and put a roof on. Structure done. (I may get the order wrong because I don't build houses) The next feature could be plumbing rough-ins. You can probably parallelize this feature w/ the electrical rough-ins and you build it from bottom to the top/vice versa but you build the entire feature. Only after that do you build the entire insulation feature, again from top to bottom/vice versa. Rinse repeat with drywall (another feature). At each of these steps you can either rush it, make incorrect decision, try to save money and do a bad job or do it with quality. You might use 2x3s for the structure instead of 2x4s. You can use aluminum wiring instead of copper. You can use R2 insulation etc. Luckily in house building there's a building code.

In software that's the same. If you build horizontally, you need to rely on things like designing and documenting APIs very carefully, build to spec etc. Basically you're just doing mini-waterfalls and calling it Agile or Scrum or whatever. But it has all of the same problems. The guy that built the xyz API last month and is now working on something else. Finally the UI people got around to building the xyz UI on top of the API and are discovering all the shortcomings in the design, the bugs that weren't apparent when "they tested the API" etc.

Build it vertically. Build the xyz feature from top to bottom/vice versa i.e. build the API and UI at the same time. The people doing it either need to communicate very well with each other or someone does it full stack. Doesn't matter. But skip the documentation and just build what you need in tandem. Build it well. You can use the same technique as with the house. The fun part in software development is that it's much easier to split these vertical slices into much much smaller parts. In the house you were only able to split the "rough in" feature into "Plumbing rough in" and "electrical rough in". In software the xyz feature can probably be split into many many more parts that can each be done in a very small amount of time. At each step you can decide whether you've been building the right thing, the wrong thing (and stop) or something that just needs some adjustments going forward. Just as in house building, you can do each of these vertical slices either with or without quality. Unfortunately there's no "software building code" w/ inspectors ;)

And if you build vertically, you can use your house err I mean software before it's finished. You can probably not move in right after the structure feature is finished. But you may, if you're fine cooking w/ the BBQ and use a porta-potty outside. You probably don't really want this if you have a family. And you don't want to ship software in this state. But after you have the plumbing and electrical done and the house is weather proof, you can cook w/ a small electric cook top, you'll have a sink and a proper toilet. You have drywall and heat. In many new houses you even opt to leave some of the features out completely in some areas. Like an unfinished basement. You can ship software that way too.

EDIT:

And even the Full stacking works in this analogy. Think "structure" part of a house. Your basement is poured concrete. That's done by a specialist usually, think of this as your DBA optimizing the SQL queries (or nowadays a BE guy w/ lots of experience in that). The framers build your walls and roof structure (trusses and such). These could be seen as your "BE guys" and the roof itself, i.e. the plywood and shingles or metal or whatever is put on by specialized roofers, these are your FE guys. Now if you have the skills and are full stack, you can build all of this yourself instead. Even if you build a whole house you could but let's say you just build a shed. Same as a house really, just everything is smaller and easier. But you can totally pour the pad for the shed (or the basement of a house but you will want to use a "library" or "framework" err I mean a company that delivers concrete by truck. You can frame a wall by yourself. It takes longer if you're talking whole house, but shed is not much slower when building alone vs. a crew. And you definitely can put on a roof. For a shed, you easily build a lean to style one yourself. You might use a "library" err I mean a company that delivers pre-built trusses for a house. And if you're building a lean to shed, anyone can lay plywood and shingles on that. If it's a whole house you probably only want to do that if you're good with heights. Maybe you're not full full stack but you can hold your own for smaller tasks.

> low code quality and technical debt is one consequence of the Agile movement.

Low code quality and technical debt were always there. The people in the early Agile movement, XP in particular, which most functional 'Scrum' teams are doing approximately 60% of, had a bag full of tricks that they used to speak to and encourage enlightened self-interest from management, and to pull a fast one if that didn't work. I can and have sympathized with this sentiment.

The problem is that when you build a system partially on trickery, your coworkers don't necessarily get the memo, and defectors engage in their own trickery to undermine what you're trying to do. A common blog post thesis at the beginning of the Trough of Disillusionment for XP was that people were attracted to the fact that XP let you omit certain steps from the development process, but at the cost of adding other steps that were much less onerous but still not particularly pleasant, and then people were cherry-picking and not doing any of the onerous bits. The analogy of eating your dessert but not your vegetables came up many, many times.

I think the next phase of improvement will have to be more transparent and more empirical about what it does. At present too many items in the Agile Toolbox are bound up in virtue and ethics and neither of those speak to the pragmatists (which is how we ended up with something called Pragmatic Programming, which, like all such titles, is more aspirational than descriptive).

We need to do more 5 Why's analysis on some of these Best Practices, rules of thumb, and aphorisms, because it's clear we're missing a few things. For instance, here's a proposition that I think makes an excellent mental exercise.

Merging code is one of the most difficult problems we have not (cannot?) solve, and lacking a solution we should avoid merge conflicts as often as possible by organizing our code to avoid them.

How many of your software practices end up serving this concern? A lot of mine do, especially if I dig. For example, grouping similar functions together, avoiding God Objects, and alphabetizing entries in unordered sets decrease the likelihood of conflicts when two people work on unrelated features. Even global shared state is really a problem of merging data from two sources at runtime. Pure functions and borrow semantics both establish a clear order of operations where you can avoid or quickly identify conflicts.

But almost always I hear these techniques described as "the right thing to do" by proponents and "aesthetics" by deniers, and so the fight continues on with no resolution in sight.

Building software is not analogous to building a house. Building software is more like what architects do before the house is built. With the difference that we can press a button and a new “house” is built within seconds.

What I often see is a chronic failure to properly define and maintain boundaries.

Chicago has only two remaining buildings from the 1893 World's Columbian Exposition: what was once the Palace of Fine Arts and is now the Museum of Science and Industry, and what was once the World's Congress Auxiliary Building and is now the Art Institute of Chicago.

In order to save costs, most the buildings were deliberately not built to last. These two were built to spec. One because its future as an art museum had been identified ahead of time, the other because its immediate purpose as an art museum meant it couldn't be slapdash; no other museums would lend their art if it were going to be housed in a potential fire hazard. But all the others, the ones that were expendable? They were immediately torn down after the fair concluded, in part so that people couldn't get themselves into trouble misusing them. And any further use would have been misuse.

I don't see the same culture of maintaining boundaries in software development. What I see much more frequently is that we make a reasonable decision to cut some corners with some purpose-specific code that doesn't need to be high quality, but fail to put up any sort of warning signs about it or otherwise prevent its reuse or misuse. When we do that, it's inevitable that someone will come along later, perhaps long after the original authors have moved to a different project, and attempt to simply reuse it rather than, say, rewriting it into something that was engineered to last.

Some attention also needs to be called to the Clean Code afficionados who spend 20 person-hours gold-plating code that was only worth four person-hours. I appreciate the desire to take pride in one's work, but that kind of misallocation of team resources does grave harm to the developer-manager relationship, and leaves the development team with little social capital they can use to argue for building things with care when doing so really is necessary.

That said, it does need to be observed that Scrum (not agile in general, but definitely Scrum) came out of the contract development community. I don't think it would be unfair to characterize the methodology as one that was made by and for people who don't normally expect to be maintaining the code they write for years to come. They ship the project, they move on to the next contract. And they work for companies where, as a matter of business necessity, teams are constantly being reshuffled around those contracts, so letting people get too specialized is somewhat antithetical to the nature of the business. It's theoretically the managers' jobs to be paying enough attention (and know their business well enough) to notice this, and reflect on whether they are working under similar conditions.

Reply on news.ycombinator.com