Hacker News Reader

Comment by sethammons | Hacker News Reader

I hadn't realized this mutation testing existed as automated tooling. I'll be looking more into it. Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed.

View on HN

> Traditionally, I've gone with "sabotaging" the code when writing unit tests; altering the code to verify a test goes from red to green or vise versa. Never trust a test that has never failed.

That's completely backwards compared to how you should be doing things.

You either write the test:

1. to verify the existence of a bug by reproducing it, or

2. to formalize the spec for yet-to-be implemented code/feature.

And then you make the test green. Retroactively writing tests for working code, only to sabotage the code... Seems like an odd way of doing things.

Some people feel more comfortable writing code without tests first, gradually shaping final design of the interface. Especially if the tests involve a lot of stubbing and mocking of things like Redis and Sphinx Search, or messing with crypto tokens, parsing HTML, freezing time, setting up global config attributes like support emails, interaction between 2 different databases, etc. Tests are code as well and oftentimes might feel heavier than production code. You can of course say that there is a way things should be no matter what but that might lead to negative emotions, toxicity and complaining instead of getting stuff done.

Agree, actually designing & implementing good tests requires a lot of effort. It's rarely wasted effort but if you really need something out the door now it can distract from short term delivery.

e.g. I'm using a 3rd party transpiler / build tool, which I don't know all the details of, my test runner wouldn't do exactly the same transpliation of code this build tool, it took me about an hour to figure out where the build tool's config file was and how to get it working with the test runner.

Did I learn more about my tooling and code base? Yes. Was it useful? well maybe not as I might kill this project shortly anyway.

That sounds like a complicated workaround for bad libraries (Redis does support unit tests?) and bad apply design with strong coupling between remote components.

You don't always write a test first. A bug? Sure, write a test that shows the bug, fix it, make it green. A very well defined something? Sure, TDD. Many times, you will write code (or come across pre-existing code) and tests come last.

Yep, TDD is quite hard when requirements are vague/changing... the reqs changed? OK now I have to rewrite my test AND the code that passes it.

Vague/changing reqs is of course a separate problem.

While in principle I agree with you, there is something to be said for 'explorative' programming- where I don't quite know how I'm going to build this feature, or I don't quite know just what the refactoring will look like, etc. Most often it's that I don't fully understand the codebase I'm working with yet, and the tests that exist are old and crusty. I want to play a little while first.

I feel like that is a valid form of our craft, and it doesn't play well with TDD. It's not the primary way that I prefer to work, but I'm certain I'll be there later today (I've got a bastard of a sprint task waiting for me).

I might be wrong but aren't these called spikes?

I do exploratory coding in a similar way, usually with a repl next to my text editor, and this first single file version is usually always trash but helps me understand the problem space. Then I write it "for real" with the knowledge I've gained.

It's entirely possible to write code that passes an as yet tested desired piece of functionality. If you want to make sure that piece of functionality doesn't get rewritten out later, you probably still ought to write a test. This test never fails though, unless you deliberately force it to.

You _could_ just assume your test is correctly written, but I personally prefer to be sure by seeing it red at least once.

I think you misunderstand.

I'm not arguing against red tests. I'm arguing for them.

What I'm arguing against is 1. writing the finished code, 2. writing green tests to "prove" the code correct, 3. trying to sabotage the finished code to "prove" that the tests works by making them red.

Sounds crazy? That was what OP said he was doing!

It's quite possible that he's testing after. It's implied, but it isn't necessarily the case.

Anyway, we all agree testing is good. And that a test you've never seen fail is bad. Personally, I try to delete exploratory code and try to use tests to drive out the _real_ implementation, but I'm going to admit that at least sometimes I'll write code that's inherently "safe", and then add a test afterwards for garbage inputs, for example.

I'm pretty sure everyone understood. There are many ways write code.

How do you know you have full test coverage if you don't mutate your code?

That’s a fair point. I don’t.

In that youre not trying to prove the tests “correct” though, but instead trying to prove good coverage.

I’d argue there are several ways to do that, and that’s certainly an interesting approach to the problem.

In either case you’ll have trouble being 100% confident though, so I guess it’s a matter of deciding when enough is enough.

Reply on news.ycombinator.com