Save the code or save the tests? (2020)
qntm.org
qntm.org
^^ comment from someone named hobbs
Exactly. Users can tolerate an app that's kinda broken in some ways, but tests don't make money. In fact tests can often cost money anyway if they're not testing what's really important. No matter what, there are going to be parts of your code that simply cannot be described by the tests that were written.
With that perspective, it's beyond silly to prefer the tests. You are saying "Should I throw away 90% of the work or 10% of the work?"
Said another way, you are asking "Should I save the chapter headings of my book or the content?". I can make new chapter headings, but writing the actual chapters is where I spent all my time.
How would you verify that algorithm?
Forall A:
add(A, 0) == A
Forall A B C:
add(add(A,B), C) == add(A, add(B, C))
Maybe throw in add(A, B) == add(B, A) if you’re angling for a promotion…
add(a, b) => a == 1 && b == 1 ? 1 : a + b
I think GP's test would probably be implemented with property testing (or even better, symbolic execution). It wouldn't necessarily take super long to run.
assertEquals(0, add(0,0))
assertEquals(1, add(0,1))
assertEquals(2, add(1,1))
assertEquals(8, add(3,5))
assertEquals(-3, add(2,-5))
it('should work', () => {
inputOutputPairs.forEach(([input, output]) => {
expect(testedFunction(input)).equals(output);
})
})
cases = [
{ in: 1, arg: 2, expected: 3, message: "base case" },
{ in: 2, arg: -2, expected: 0, message: "negative numbers" },
...
]
for ( c in cases ) {
actual = obj.do( c.in, c.arg )
assert( actual == c.expected, c.message )
}
So to rephrase the question: save the code, or save something that makes it easier to change code you no longer have? It seems a bit easier to answer if asked this way...
Even IF I ever wrote unit tests for it - which I don't do and have never done for anything - those unit tests wouldn't have anything in them from which I could reconstruct the UI.
No. You save the code and the surrounding UI and the UI glue code.
Surprise, common sense.
Saving the tests is like needing to know the answer to the Ultimate Question of Life, the Universe, and Everything (which is 42) and discarding the question.
And honestly, the 0 test coverage base was actually easier to deal with.
Tests are good, but like all code they can be implemented poorly.
Ofcourse I do realise that in real world examples there's not many test cases which cover a feature fully. But this was all thinking hypothetically.
Having test code and dev code in separate repos is a pattern in some places. Where that exists, save the dev code. Any test code that's not allowed to mingle with dev code is highly suspect and almost certainly not a working spec.
I just refuse to write tests. I think it is complete waste of time.
Here, I said it.
My preferred style of working is writing something and then refactoring it to my satisfaction. I toy with interfaces as I discuss meaning of various things, learning in the process. I am mercilessly cutting stuff that is not necessary, finding ways to make everything shorter, simpler, easier to understand. In the process I am constantly moving stuff around, changing concepts for other concepts, removing code that has already been written that I find no longer necessary, etc.
Writing tests might make sense if you know how the code will be structured, what is going to be exact contract for that method, and so on. It doesn't make sense if you are going to be changing it in a moment, multiple times, with 20% probability you are going to remove it anyway.
I tried to give it a shot, but writing tests breaks my flow as I try to put some kind of structure as quickly as possible and then when I am polishing it to perfection.
Writing tests after the code has been written seems like a waste of time for another reason. I have already written the code and thought through the contract. Writing complete unit test suite would take about as much time as time necessary to write the code in the first place. But if I put so much time writing that code correctly there is relatively little chance somebody is going to be reworking it. That's basically the whole point, to make it less likely that it is going to be hotspot of work in the future. I put effort in that module so that I don't have to go back to it.
Spending a lot of time now to have some probability it is going to save some amount of time in the future, most likely less or comparable with the amount of time invested in the tests -- total waste of time.
Later, assuming I actually try to refactor code that is tested, the tests are getting in my way. Thanks to tools like IDE I can edit a lot of code efficiently and with high degree of reliability. I can move stuff around, simplify, then move again, then simplify again and so on. But there is still no automation to correct unit tests accordingly -- every time I changed the structure of the code significantly I have to go to correct those tests.
So it kinda seems it doesn't help me with the refactoring task either. There might be a chance I make a mistake that could have been caught by a unit test -- but my experience shows that it is rather rare occasion that a unit tests prevents me from doing something stupid.
Instead I am spending effort now to edit all those unit tests for a speculative benefit of saving some time later, and likely less than I am spending now.
Add to that that most codebases I have seen in the past consist mostly of bullshit unit tests -- tests that are there only to prop up test coverage to a level required by CI pipeline and usually test only simplest parts of the contract or even plainly stuff that is completely self explanatory.
Why the bad state of unit tests? I think reasons are quite simple:
1. If the test is written after the code it means the developer has already written the code and is now not that much interested in doing something else that doesn't seem to be immediately improving the product. About the same reason why most documentation is shitty and out of date.
2. Production code is constantly being tested and there is feedback if it is not correct. There is no feedback if tests are not correct. The only real requirement for unit tests is that they pass and that there is enough of them.
3. If the deadline is looming, what do you think is getting cut first? Expect absolute least effort into writing tests or documentation.
4. It is hard enough to get developers to review production code. I have yet to see a reviewer that would do anything else than skip or only gloss over test code.
Having to make a choice -- do I want more well written code or less code but with test coverage that is most likely going to bring little value... my choice is rather simple.
That said, when making incremental changes to an existing project, tests (and in some cases TDD) have a lot of value. There comes a point where the project changes from a creative endeavor to a product. Then you need tests.
The problem is how you create a test suite at that point in time without being negatively influenced by having written the code. Most unit tests I've seen in the wild are BS for this reason: the person writing them just repeats the code they wrote to implement the feature, or at least makes the same bad assumptions. In most cases they mock out so much that the test bears no relation to the real world, and breaks with the slightest change to the code.
For this reason I prefer to write tests from the top-down instead of bottom-up. Start with a few end-to-end tests, making sure to cover the happy paths and a few of the more probable error cases. These few initial tests are more valuable than any that will come later, because they best correspond to how the product will be used, and they are least likely to break during later refactorings since they don't depend on any internals, making them very cheap to maintain. For smaller projects, these tests may be all you need.
Every level you go below that incurs an increasingly large maintenance cost, so you have to weigh it against how often the design of that component will change.
When I do write unit tests it will be for essentially pure functions where the domain of inputs is sufficiently small that you can do fairly exhaustive testing. For example, a function that determines whether a floating point value can be losslessly converted to a 32-bit integer is perfect for unit testing. Nothing needs to be mocked - whatever the code does under test is what it will do in production.
All of this is my preference, but the only thing that really matters is that you actually use your brain when deciding what tests you write: weigh up the costs of the test against the set of potential bugs that it can prevent. The real sin of TDD is that it teaches you to write thoughtless tests.
With tdd the idea is you write a test first to explore how your code should be called, then you make that code, satisfy the test and add more requirements and repeat the whole thing. Once you have a bunch of tests which are defining the use cases in your code, you can refactor/structure your code internally without affecting the tests or if you think the whole api could be simpler, change the whole thing along with tests. The refactoring at this stage is generally of higher quality since you have a concrete idea of all the ways your code needs to work while as if you refactor early there is little guarantee that it's any good except for intuition.
If you are structuring or abstracting without examples of how your code is to be used, then the design often turns out to be inefficient, overly complex, coupled or over abstracted.
Admittedly this TDD approach is a bottom up design method, it takes time and effort and it's not well suited for problems where you already have a good idea of how code should look like/feel/behave.
You don't need to run these tests every time in ci, you can delete them or keep them as documentation or run them whenever you have to work on the code again.
There are other ways to get a stable system. It seems like the code that you're working on doesn't see much code churn and you have a good understanding of the system. This is definitely a situation where I see testing not being as valuable.
I think one of the big reasons testing is considered a good practice is we're relying more on dependencies that need to be updated somewhat frequently, and semver is not taken as seriously, especially in certain web ecosystems. There's a decent chance you don't work in such an ecosystem where it's not an issue, but that's the biggest reason for me to advocate for testing.
The other big reason is that it's easier to make changes in a codebase that you aren't familiar with when tests are available. This is especially true with software that may have non-obvious corner cases.