Comment by programminggeek | Hacker News Reader

Comment by programminggeek | Hacker News Reader

Parent Full thread

programminggeek

What DHH describes is called the test pyramid (http://martinfowler.com/bliki/TestPyramid.html), which despite the fact that DHH seems to dislike Uncle Bob, it is something Uncle Bob has demonstrated and advocated multiple times in very sensible ways.

The bigger thing DHH doesn't mention or explain is that Basecamp is designed to be intentionally simple. Much of the complexity that you get in client work where you don't control the requirements Basecamp is able to simply avoid by not building.

The idea that if you own your own product and control the requirements that you can have a MUCH cleaner codebase without needing as many tests is 100% true. But, David didn't say that and it's a shame. It's a much more powerful point because there are cases where clean architectures are valuable and the standard Rails MVC pattern isn't enough. At the same time, there are plenty of times where the basics will get you a very long ways. If you control the complexity, simple MVC might be all you ever need.

Everything in software development involves managing requirements and tradeoffs. If you simplify requirements, you reduce complexity. You reduce complexity, you reduce surface area required to test.

It's like weight reduction in cars, you reduce overall weight a few pounds, you can also reduce an extra 5-10 lbs. in required components to support the added weight/complexity.

I just wish that people could discuss these ideas with nuance and empathy towards other project's requirements. Some projects require a lot of tests and have significant inherent complexity, some don't. A one size fits all philosophy might sound appealing, but it doesn't work.

Don't throw the TDD baby out with the bathwater.

I wish this post stays at the top of _ANY_ discussion about automation testing.

Couple V.E.R.Y important points:

1) Testing Pyramid

People, who agree or disagree, talk a lot about unit-testing yet they never talk about the other types of automation testing; as if the rest were forgotten.

To make matter worse, people often haven't had the experience of managing full-blown System Test. Hint: it's very brittle, slow, and expensive (even with the improved infrastructure automation tools such as Chef, Packer, Docker, Vagrant). Don't forget the magic word "tightly-coupled" that'll demotivate you hard and quick whenever you want to make some changes.

2) Control the requirement, control the codebase

This is something that NO one ever mentioned in ANY "clean architecture/codebase" discussion. People often focus on the tools (Rails, Haskell, Clojure) and never mentioned that they actually control the requirements (or in another words: opinionated) or the requirements take a back seat against cleaner code.

For example: if it didn't fit with Rails, don't do it <= controlling the requirement.

I would prefer this post stays as an example of how not to do a post about the testing pyramid. Martin Fowler post/article is better IMHO, and I'm sure there are much better posts.

There are some good ideas mixed with fallacies and disdain/contempt. These are not good in any discussion.

Another thing I think DHH has missed here is that his experience is not particularly broad. His achievements are very impressive, but if I recall rightly, he started Rails before finishing his CS degree, and that's basically all he's done since.

One of the core assumptions of Rails is having a database. A lot of it is focused on turning HTTP requests into SQL requests, and turning SQL responses into HTML. Which is fine; that's what a lot of web stuff is, and it's fine for that. But it's not the only thing that happens in the world.

I've TDDed systems that are not locked in intimate embraces with databases, and TDD is much easier and more pleasant there. On one project we did like that, we ended up with 40k lines of test (and a similar amount of production code) that ran in a few seconds. It was the most pleasant development experience of my career, and TDD was no problem there.

It's a lot easier to write unit tests for, you know, units. For example, if you have a date-to-readable-string conversion library, you can test the crap out of it since it's input is a date object and its output is a string. It's a unit of code, so all the testing that's required is unit testing.

On the other hand, when the "unit" you are testing relies on six external API's and your own database to work, best of luck to ya. It's not a unit at this point, it's the glue that holds other units together. It is important to get it right, but given how many different combinations of failure modes there are you are not going to test it all and you are not going to get it right. Mocks/fakes do not make this stuff much better as timing and latency are important too. What if the UPS address service you are using doesn't return an error but times out? Now your production code has to include enough of a framework for simulating network errors.

I do agree with TFA: TDD is a niche thing that has been so widely embraced for all the wrong reasons that it's no longer useful even in the niche. Also, system-wide testing is very important as well and the tooling for it sucks. My rule of thumb is to be pragmatic about automated testing: test the things that will give you the greatest ROI in terms of time/effort/time-to-market/etc. but don't be religious about it. The tests are not the end product and should not be treated as such.

Agreed. Unit testing requires units.

I also agree that being pragmatic is generally the way to go, but that kind of pragmatism has limits. If I'm too focused on short-term pragmatism, I will let the issues of external systems infect mine. If I'm being longer-term pragmatic, I'll try to keep my code relatively clean and reliable, no matter how kooky the things I'm integrating with.

For example, the first REST integration I do, I may just say "we'll ignore network errors and see what happens". But if I'm doing a few, I'm going to look hard at extracting a reusable layer of code that handles layers, timeouts, partial responses, and the like. And that I will test properly. That way the rest of my code can assume sanity.

Short-term pragmatism and long-term pragmatism. Excellent concepts that I've never thought to quite express.

It seems you can always 'win' an argument in software engineering or TDD by talking about taking the 'pragmatic' solution, or the 'proper' solution. It often seems rather arbitrary whether to build something properly now, or introduce some short-term technical debt.

It's easy to accuse people of being cowboy coders, or architecture astronauts, and it unfortunately seems to fall into these dichotomies. Admittedly there are people like that, but mostly through lack of experience. The progression is maybe commonly cowboy coder-> architecture astronaut -> balanced software craftsman.

There's probably a bunch of steps missing on the way and it's not a clear linear path either, but I do like the term of having different durations of pragmatism, than a one-size fits all, "We'll cut corners to be pragmatic, because we're not architecture astronauts".

That path you describe was definitely mine. I think the cycle driving that change is something like:

Things seem great -> I'm ignoring a problem -> Ok, there really is a problem -> I hate the problem -> Look, a solution! -> The solution is the best thing ever! -> Ok, I've taken it too far -> Things seem great.

Regarding the arbitrariness of building something right versus taking technical debt: to me that's great grounds for experimentation, as long as I can trust business stakeholders to give me room. At my last startup, we sometimes took on substantial amounts of technical debt, because I knew could always trust my cofounder to give us the room to clean up our messes when it became necessary.

But in a more pathological business setting, I'd be an absolutist. No technical debt! Never! Because I've seen too many places take that long slide from a little technical debt to an enormous low-productivity snarl. And then they just accept that as normal, generally for the rest of the company's life.

So I think a number of these engineering arguments are framed too narrowly. People often end up cowboy coders or architecture astronauts because that's what's working for them in their circumstances.

Reply on news.ycombinator.com