Why Software Projects Are Terrible And How Not To Fix Them | Hacker News Reader

Why Software Projects Are Terrible And How Not To Fix Them

sealedabstract.com

136 pointsdrewcrawford23 comments

A couple years ago, when I was at a previous employer, I was lamenting the typical corporate frustrations to a friend. The executives seemed to have a new "strategy" every quarter. All our projects had unrealistically given deadlines. Even when we successfully released a new product, it was basically DOA, because they were never things we would use ourselves. It was basically impossible to pivot our aging flagship product, because it has grown so complex that it was impossible to get any more incremental gains without a huge commitment in resources. At least once a month I'd be asked, "Well can we make this change to the product quickly?" and I would always say, "No. There are no quick changes anymore. You can either commit the staff to getting the product back into a state where we can do quick changes again (ie. refactoring some of it), or you can just put the product in maintenance mode and have the staff do something else." And yet no matter how many times I explained why, using as many MBA-friendly concepts and analogies as possible, they never understood.

My friend asked if any of the executives and managers had an engineering background, and I literally laughed out loud. Of course they didn't. If they did, they would have understood concepts in this article. Instead they were exactly like that political douchebag in The Wire video.

One of my greatest hopes for the current proliferation of startups is that some of the technical talent there actually does end up as leaders corporate America. I see the fundamental problem of corporations led by execs with the classic MBA-educated, McKinsey/Bain-trained background is they've never created or produced a damn thing in their lives, and just fundamentally don't understand why their decisions are so dysfunctional.

If not understanding you is the worst that happens I'd say you're lucky, but watch out...

My buddy and I have been resisting "quick changes" on projects that have a lot of technical debt for a number of years, and what has happened is that managers have just gone to some other less experienced developers who have no scruples to get their changes done without the "fuss". Our reputations have taken a bit of a hit as well - we're "too careful" or don't see that the "perfect is the enemy of the good" etc. That's what they say in front of us, I'm sure they're more colorful when we aren't listening. I've learned to mitigate this by working with a couple of managers that aren't clueless but it's a danger when that isn't an option...

Technical debt is a major red flag especially in a savvy organization that realizes it's accumulating it. Especially if much of the early success relied on it.

I was talking with a prospective client that had spent the past three years rushing a SaaS solution to market. They even referenced the technical debt as an area they need help with overcoming as they move forward.

Second order change is difficult for any organization or established project, but remaining consistent with your message can drive the process in that direction. Sounds like a challenge for the two of you, but try to bring solutions / recommendations instead of minor resistance that might get misinterpreted. Not easy I know.

How do you spend 3 years rushing a product to market? What type of system were they building?

This company operates in the e-medical records space and works with hospitals, physicians, and other providers. It is more than just records management and deals with referrals for service along with analytical capabilities etc.

Hard space to penetrated but with a lot of federally funded initiatives the past few years. New regulations around EHR is driving the space.

I've reached the point where I refuse to enable such people, even if it means short term sacrifice for myself.

They need to die off, in a career sense. Any help I give them, even when well qualified and executed on my part (i.e. despite them / their attitude) is just adding to the problem.

Seriously. There are a lot of "morons" and "jerks" out there who need, simply if painfully, to be "disenabled".

(And, to some extent, if their/the customers suffer, well, those people voted with their feet in becoming customers. In our "free market" society (this is a U.S./"Western" perspective), it's somewhat akin to our politics: You can bitch all you want about the politicians and how they are screwing you over, but YOU elected them.)

>Turning around a software project is like turning around a tugboat: it takes a lot of time and energy, and tremendous momentum.

Off-topic sailors nitpick: tugboats are highly maneuverable, what takes a lof of time and energy is turning around a very large container ship with a tugboat

See for example: http://www.youtube.com/watch?v=rkCqDLR2s78

Indeed few boats that size are as maneuverable as a tug. It's a good metaphor for a skilled developer: even with that kind of agility, it can take a while to change the direction of something with a huge momentum. And a software project has momentum vectors shooting off in high dimensional space.

You can't really fix things like software publicly. That'd never fly in a typical company, startups excluded of course. The only thing you can do is do it secretly and there are practically two ways to accomplish that.

The first way is hard to do if there's a large group of programmers working on the project but it's very suitable for older projects that is still under development but with lower staffing.

If possible, do things efficiently locally (use source control, use python/jython, generate code, do automated builds, do automated testing, reuse, and only deliver what you must) but leave your advantageous practices unadvertised. Then, slowly abuse your edge to steal maintenance and refactoring time from actual to-be-done tasks and features. Eventually you can ease your own future work and you'll get more time to work on the system instead of the features which become easier to add.

The second way is to employ a targeted attack against a whole malfunctional group or team at once: skunkworks projects. Do the damn thing from the scratch and do it right, then replace the old steaming pile of junk with it before anyone gets to say no, and weasel your way back to the surface by toting things were just changed a bit, and never replaced.

Of course, this comes with the assumption that the evil is in the system and your colleagues aren't actively demolishing everything you build.

     Do the damn thing from the scratch and do it right, 
     then replace the old steaming pile of junk with it

This reminds me of the many attempts to rewrite nethack in a higher level language, like ruby or python.

They all fail miserably, leaving nothing but a smoking crater and piles of code that is never to be used.

Turns out all those millions of lines of c code written over a period of 25 years actually do quite a lot.

Yes. Programmers (myself included) constantly want to rewrite form scratch: But this time, we're going to do it right!

Unfortunately, often, you'll draw exaggerated conclusions from past failures and over-engineer things ("All problems in computer science can be solved by another level of indirection."). You might also end up creating new design problems, e.g. a bottleneck in an area that the old design just happened to avoid. And even if you do get everything right design-wise, you'll always end up creating new bugs along the way.

(Obviously, none of this is a new insight, the pros and cons of rewrite from scratch have been discussed back and forth forever.)

Yes, this point of view is well known and well documented.

But the opposite may have some weight too. I think I read somewhere that Google codebase had been rewritten from scratch. Maybe it was early enough, because I suppose rewriting it today would be painful. Are there any other successful rewrite from scratch stories? Maybe vim?

Google doesn't "rewrite" things in the traditional sense.. they develop new versions of old systems when the old system has reached the limit (or is just grossly inefficient). The new systems generally are not at all architecturally similar. I don't think I've ever heard of a rewrite "because the code was messy" or anything other "it can't do X and we need X".

I feel secretly rewriting software from scratch is rarely the right solution. Especially if management has already decided thats not the direction they want to take.

Your job as an employee is to make your boss happy. If you don't have political buy in from your boss then he/she isn't going to be impressed that you have been spending all this time on this project instead of spending that time on whatever it is that is important to them. Even if its overtime.

If you disagree with your boss that radically, then you should find a new boss because you won't be successful in your current position. And also because you won't ever be happy. Find a boss who you get along with. And be happy.

I take a different approach, mostly because I've fought hard to do the right thing..and it rarely works..and I usually end up looking like the bad guy.

I do what my managers want and when things start falling apart, I remind them about the warnings I gave them before-hand (sometimes with a smile)

I don't worry about the business-side of things because I'm not a partner in the company and it's not my job. If these situations happen too often, or I get blamed, I leave and find another job (usually with better pay).

I saved my last company from many disasters because the boss decided to let his 20 year old wife with absolutely no experience make business and marketing decisions based on what she was feeling that day. It was actually pretty interesting, because I used it as kind of a testing ground for the worst ideas ever. In many cases, the results were predictable and entertaining.

In 2008, we saw that things were getting bad. Orders were slowing down and management asked me to give them some marketing ideas. I gave a bunch of ideas that were inexpensive to implement, but would take time. They didn't listen. Fast-forward to 2010. They want to start trying my ideas. When they weren't making money in 2 weeks, the ideas were considered worthless and dropped.

Sometimes you just have to let children touch a hot iron to see what it's like to get burned. The smart ones will learn from this and not get burned in the future.

I think some of the other replies don't quite get your do-it-in-secret concept, but I completely understand what you imply, it isn't so much that you're advocating a complete rewrite of a codebase (or at least, not just that).

I think you're talking about entire build out and a conceptualization of an entire system that comes from a single narrative, and is thus, coherent.

I have witnessed this in others and done this myself in various projects, and you could say it was a hack, or maybe it was making a whole bunch of decisions with various scripts or procedures, but without letting anyone know that it could be an issue until you had just went and done all of it because you were simply too tired of all the bitching and other (actually really great) competing suggestions. I think that's perfectly fine if it moves things forward, and can be a great help to a climate of dysfunction, but you can't fuck it up, so it would be like a very very last resort thing possibly.

Original source of the dilbert comic in the article:

http://dilbert.com/strips/comic/2006-11-05/

dilbert can be painfully accurate, to the point where you have to stop reading it to avoid depression.

People who have too many problems dump them onto someone else. Such people in management make life miserable for anyone below them. Problems tend to move around in circles and pointing out an issue is a good way to have it boomerang back to where it becomes yours to solve.

It's much better if you have coworkers and especially management with whom you can cooperate. If there's at least enough flexibility to improve things, it's possible to have things snowball in reverse, as it were, where each person tackles a piece of the problem and it gets taken care of.

But it's hard if you work with people who don't care. For example, it can be frustrating to get blamed for causing problems that have always existed, with no one noticing them, simply because people were too far away from the consequences and those who suffered the consequences had no idea about their cause.

And yes, even simple things like "source control" can be a tough sell when the devs insist that keeping the .c file in the folder with the .exe counts as source control. For a nearly 30-year-old DOS program, written with ASCII menus (think printf, not ncurses), uses only statically assigned globals (malloc? what's that do?), deeeeply nested ifs, that directly writes outputs to hardware I'm not sure anyone at the company still understands, and there's a custom build for each install? I'm just grateful I have nothing to do with "maintaining" that source and that DOS makes it easy to do backups via xcopy and a network drive. In retrospect, I might have been happier not knowing what was in those .c files.

UX comment on this website: I like to click and highlight words as I read. I know I'm not the only one who does this. On this site when I do this, the article keeps drawing focus to the top of the page and its...annoying. Why would someone do this? I've never experienced this issue on any other site before.

I thought I was alone in that preference! A designer was watching me read through an article and he was like ".... why are you clicking everywhere?" It helps me keep track of my place, and I'm ready to control-copy even faster.

There is a strong correlation between terrible projects and the value that the project will deliver.

The lower the future value and the uncertainty that it will bring the value, the more problems the project will experience.

So, if you as a developer ask "how is this going to deliver value (make money)?" and you don't get a convincing and logical answer the more problems you will experience.

The easy part is, the bad projects are really easy to identify because no one on the project can tell something argumentative.

- "You have to build a video website in 1 week!"

* How is this going to make money?

- We will charge every user for every video they upload

* So, it is like Youtube but paid. Why will people use this instead?

- BSOD...

Has this guy ever even read Joel's stuff?

The acronym TDD keeps popping up time and again.

Joel's not a TDD fan last I heard him talking about it (admittedly stackoverflow podcasts a couple of years back). He certainly wasn't a fan of it when the Joel test came out because that pre-dated TDD.

Fact is, while I agree with some of the points in the article, there are things that aren't accepted practice and TDD is one of them.

Personally I think it's a colossal waste of time apart from some complicated methods.

And don't even get me started on the esoteric 'scrum' or 'agile' where everyone has their own pet definition of what it really means.

He never said or implied that Joel is a TDD fan in the article. He also concedes that TDD might be a controversial practice in the first paragraph. In any case, the merits of any particular practice have little to do with the main thesis of this blog post.

Reply on news.ycombinator.com