Consultants Ate My Unit Tests

I had a company license the source code from one of my old projects to use as the basis for a new site they were building. I built out the bits they needed on a consulting basis and got them launched. Then they started thinking about how expensive I was, and how much they would save if they moved the development overseas.

Fair enough. My philosophy on such matters is that it's best to let a client learn this lesson through painful experience. All the best, guys.

They made it about a year before the site stopped working completely. Reading this article reminded me of the first time I opened the source that came back. You just wouldn't believe that you could do that much damage in such a short time.

I quoted them double my old bill rate and got them back up and running, with all the new functionality they had been wanting, and re-launched. They brought in a new guy to run things, and he started making comments about expenses, and how surely there must be a way to do this cheaper. He even found a local kid who knew some C# and would be willing to help out for a lot less than I was billing.

No problem. All the best.

I took a quick look at the codebase when they got back in touch a few months later and politely declined to come back on board.

Thing is, once the PM was informed what they had done we did a couple of sprints where we restored the unit tests (which we still had in our history) and got them passing. They had no idea the consultants were doing such shoddy work in order to meet deadlines. The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted - and that's for all his projects not just that one particular projects. Now that the story is part of our IT lore the new consultants get the message loud and clear we're not afraid to send them packing.

In the end this turned out to be a good story for us - but wow.

Maybe the tests would have had some benefits, but would they have enough to justify all the extra time to write/maintain them? Certainly not.

In my experience, good programmers write good code, tests or not, and bad programmers write bad code, tests or not. I'm not against testing altogether, it makes sense in some contexts (like very widely used libraries). It's also useful for beginners who aren't sure "where to start" with a project. But even in cases where people swear that their tests are invaluable because it's tricky code that breaks often, that's a sign to me that that code needs to be made less fragile, not more well tested. TDD is a cargo cult.

I didn't spend much time consulting for them.

- Often the quality of unit test code is a lot worse than the production code.

- Some times people test too much in a single test (ie. more than one unit) making it very hard to change the design of the application.

- Some times a unit test misunderstands how the application is supposed to work or the requirements change.

- Some times a unit test tests something too little or too trivial.

- Some other test is already covering the case.

I personally don't weight 100% test coverage as critical. Often other aspects of QA in application development are more important (i.e. integration test or documentation).

The project manager in the room was worried about the time it was going to take. All the developers in the room piped up to say "oh, no, this actually saves time"

The next day, I found myself in a private meeting room with the project manager and my line manager.

Both were trying to tell me that unit tests are not part of the scope of the project, not were they approved and, that I was to stop writing them.

Manual QA was the way to go at the end of the project.

Yesterday, I was in the office from 10am to 2am on a national holiday addressing issues arising from QA.

It was unproductive to spend the day wondering how many of those defects I'd have been able to catch months ago if I'd been writing tests all along.

When you're the only one on your team who respects the tests, the world feels like a very bleak place.

But they're not. Do we assume business owners are dumb? Do we assume managers are dumb? That's a popular position for technical people. "Oh, you just don't understand how valuable those unit tests are!"

But the cost of maintaining unit test vs. the cost of fixing bugs later might favor just fixing the bugs later. Maybe it doesn't sometimes - but one is an upfront cost you know about, and one is a nebulous cost that you might or might not have to pay later.

Given that, I know what most people paying the bills are going to choose.

2 years ago, I was at an entirely different company. They handed me their test plan documentation. Lo and behold, it was my document, now laden with another consulting company's logo. It was missing a few tests, but there was nothing new.

I was angry for a solid 4 hours. Then I had to laugh about it. I would be flattered that my 18 year old work was still being used, but it was quite simply dated. If a big company was being cheap and getting things built by offshore resources who had no better way to accomplish it than by stealing my work, then they got what they deserved.

Expertise in software maintenance isn't common and it isn't cheap. Every greenfield dev is going to eventually see his project handed off to people who won't understand or respect his conventions and methods. That mostly destroys it, but business requirements can put it on chemo and a food tube and keep it alive against its will for a long, long time, and it's not a pretty sight.

It's practically impossible to. I've been maintaining an ancient Rails codebase for the last two and a half years. I'm perhaps the fourth or fifth such guy to do so since it was built. The project was never updated past a certain point release and it's more work than it would be economical to do to do that now. I'm experienced enough now to take it on, but both me and the company have better things to do with our time and resources.

It is so ridiculously easy to make a bad decision that seems like a good decision that instead ultimately dooms the project. Software is like this weird life-form from another part of the galaxy whose motives are so alien to us that we can't even begin to come to grips with it. We know it's not actively plotting against us, well, pretty sure, but more than that? All you can do is shrug your shoulders and hope it doesn't consider you a threat today.

Outside developers don't have any real ownership of their work product. Even if they know better in principle (and many do), they don't have any incentive to practice good stewardship. And bargain-basement outside developers aren't magical productivity pixies; the reason they are able to charge lower prices because they don't need to charge you for all the things they aren't taking the time to do.

Unfortunately, many execs without development experience neglect to see it this way.

They see "expensive cost center" that can easily replaced with cheaper labor. Cool thing is, as others (like the author) have said, they tend to get burned and end up re-hiring you on even better-than-original terms.

Kudos to the author for doing good work!

You've got the "large scope" unit tests that test functions that take lots of dependencies, via mocking. These often end up being so mocked, that you're testing the mocks more than the function itself. You feel smart when you write it, but a week afterwards you can't even remember what it's testing.

You've got the "small scope" tests that test small pure logic functions. Great, but these hardly ever change. It doesn't seem worth the overhead to maintain a test for unchanging code.

There's a "fuzzy middle" that could be reasonably worth unit testing, but these are the functions that are most likely to get completely ripped up during refactoring and new feature dev.

To me, a sound architecture (no global variables, a minimum of mutation[1], etc) and a sound type system means things generally work the first time anyway, without the need to maintain a big honking test suite and everything required to run it.

I'm finding it's much quicker to get things out the door without this overhead, the architecture ends up being simpler (far simpler in some cases) because I don't have to put test hooks in for everything, I can refactor without the worry of how many unit tests will break, and I've experienced no downturn in quality.

Of course there are cases (public core libraries, large distributed dev teams, really complex pure logic) where unit tests are absolutely necessary, but for most of my projects these days I'm finding them to be redundant.

[1] In fact, one could argue that the greatest benefit from the TDD phase was that TDD is harder with globals and mutation, so it reduced incidences thereof.

I know of one consulting shop that had a dependable stream of income from a major European bank, because that bank had a project that didn't use source control, so they got to be the periodic hero and get paid for it.

In one project next developer ate my build script. Why bother with compiling less to css if you can just edit css and forget the less files and build script. Same went for r.js bundler.

In other project I participated angular got dropped because people couldn't be bothered with it.

>> But maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)?

> This is implying maybe they intentionally disorganize the code so they'll get more billable hours?

> I seriously seriously doubt it.

> Rather, it's as simple as: If you want to add features or make changes paying the cheapest possible amount, you simply aren't paying for well-organized code.

> It's not just less experienced programmers, it's less experienced programmers trying to get you the feature in the least possible hours. They aren't doing artificial things to bill you more hours, it's in fact quite the reverse -- they really are trying to get the feature done in as few hours as possible, to bill you as few hours as possible.

> And when you do that, you simply don't have time to organize the code well, or keep the tests working. You're shoving it out the door as quick as you can. Of course it takes more time to keep the code well-organized than to just hack it until the feature works as quickly as possible. Of course it takes more time to maintain the tests -- over the short term, and all you ever have is the short term when you're paying and getting paid little by little feature by feature.

> They have other customers, they don't need to bill you artificial hours. They really are billing you as few hours as they possibly can -- because that's how they get customers, being the cheapest. That's exactly what the client asked for, it to be done as cheaply as possible.

> You get what you pay for.

I have only recently started working for a consultant that bills hourly. It's an eye opener. We are a small shop of experienced developers, we work for people that do want us to produce quality, and we do, we do good work, and we do write tests and maintain our tests. But it is a still a constant struggle inside my head between doing it as right as I really want to, and not charging the client more hours than the feature seems 'worth'.

> If you as a client prioritizes price even higher over quality (and you have no way to judge internal quality anyway, not being a coder, so why wouldn't you? It looks nice and works, what else can you judge?), if you're hiring less expensive developers to save money... they might be skilled devs in fact, but if you are trying to get it done as cheaply as possible, you are not going to get well-organized code. And you as a client don't care, what do you care about well-organized code? Until it reaches a breaking point where your technical debt is so high you can't get any more features at all, and you realize, oops... and probably still don't understand what you did wrong or could have done differently.

> Good software is expensive. Too expensive. More expensive than most people who need good software can afford. Which is why we have so much shitty software.