Hacker News Reader

> if you are good enough to write perfect tests for your code, just write perfect code. If you aren’t perfect at writing tests, how do you know the tests are complete, bug free, and actually useful?

This sentence makes no sense. Tests are infinitely more straightforward than code. I always go back to my dad's work as a winder before he retired:

After repairing a generator, they'd test it can handle the current that it's expected to take by putting it in a platform and... running electricity through it. They'd occasionally melt all the wiring on the generator and have to rewind it.

By your logic, since they weren't "good enough" to fix it perfectly, how could they know their test even worked? Should they have just shipped the generator back to the customer without testing it?

No, they often aren't, and UI can be complex to test.

> if you are good enough to write perfect tests for your code, just write perfect code. If you aren’t perfect at writing tests, how do you know the tests are complete, bug free, and actually useful?

This was the original quote. Just because one aspect of a system is difficult to test doesn't make this quote true.

> No, they often aren't, and UI can be complex to test.

It can be but I've seen different ways to get confidence through testing. Just because it's complex doesn't mean we shouldn't do things to improve our confidence that what we're building works.

Go back to my example above: I guarantee that a repair has been tested, returned to the customer and failed the test after it was fitted. Why? The fault was elsewhere. But testing gave him confidence the generator wasn't the source of the problem.

IMO if your implementation is that unstable (you mentioned the UI/workflow changes fast) it isn't worth writing a test for it, but also, I don't think it shoud be released to end-users because (and this is making a big assumption, granted), it sounds like the product is trying to figure out what it wants to be.

I am a proponent of having the UI/UX design of a feature be done before development gets started on it. In an ideal XP/agile environment the designers and developers work closely together and constantly iterate, but in practice there are so many variables involved in UX design and so many parties that have an opinion, that it'll be a moving target in that case, which makes development work (and automated tests) an exercise in rework.

I think there's a great balance here in these environments:

- write tests as part of figuring out the implementation (basically: automate the random clicking you're doing to test things anyways)

- Make these tests really loose

- Just check them in

Being unprecious about test coverage means you just write a handful of "don't blow up" tests for features, that help you get the ball rolling and establish at least a baseline of functionality, without really getting in the way.

Chiming in as an end-user of software: please try to minimize the amount of times I need to re-learn the user interface you put in front of me.

Aaaaaaand I replied to the wrong comment, mea culpa!

I agree with you that the ideal is to have UI/UX work resolved before starting dev work.

In my experience, this has never happened. I’ve moved around hoping that somewhere, leadership has fixed this problem and nope. It never happens.

There are just too many unknowns and never enough time for design to stabilize. It’s always a mad dash to collect whatever user info you can before you slap together a final mock-up of the interface and expected behaviour.

> Somewhat hyperbolically and sarcastically: if you are good enough to write perfect tests for your code, just write perfect code. If you aren’t perfect at writing tests, how do you know the tests are complete, bug free, and actually useful? :)

Well obviously, you just write tests for the tests. :3

It's called induction.

Qui testet ipsos tests?

> Well obviously, you just write tests for the tests. :3

I had a friend whose first job out of school (many years ago) was part of a team at Intel assigned to write tests for their testing tools. When it matters enough economically, it will happen. As you can see from recent news, that is still not enough to guarantee a good result.

It's actually called mutation testing. And luckily it's almost fully automated.

> if you are good enough to write perfect tests for your code, just write perfect code.

I have yet to see anyone claim they write perfect tests.

> If you aren’t perfect at writing tests, how do you know the tests are complete, bug free,

I never claimed to produce or seen complete tests. I never claimed or seen bug free tests.

> and actually useful? :)

I know that whenever I fix something or refactor, test fails and I found a bug in code. I know that when we do not have have the same bag again and then again the same bug and again the same bug.

I know when testers time is saved and they dont have to test repetitive basic stuff anymore and can focuse on more complicated stuff.

watwut wrote:

> I know that when we do not have have the same bag again and then again the same bug and again the same bug.

Well, username checks out :-)

Types are there to ensure against human error and reduce the amount of code we need to write.

Tests exist to guarantee functionality and increase productivity (by ensuring intended functionality remains as we refactor/change our code/UI).

There may be cases where some tests are too expensive to write, but I have never come across this myself. For example, in functional tests you would attempt to find a secure way to distinguish elements regardless of future changes to that UI. If your UI changes so much between iterations that this is impossible it sounds like you need to consider the layout a little more before building anything. I’m saying that based on experience, having been involved in several projects where this was a problem.

Having said that, I’m myself horrible at writing tests for UI, an area I’m trying to improve myself, it really bothers me :)

Tests can be expensive to write if they are an afterthought, and/or the code is not written in way that is easy to test.

UI tests can be cheap but they require some experience in knowing how to write a testable UI. One way of achieving that is writing them as early as possible, of course. Which is not always possible :/

> the code is not written in way that is easy to test

Which isn't devoid of downsides either

That's a good point. Sometimes more ergonomic APIs can be harder to test.

> the UI/workflow either changes so fast that you don’t write tests for it

This is my number one pet peeve in software. Every aspect of every interface is subject to change always; not to mention the bonanza of dickbars and other dark patterns. Interfaces are a minefield of "operator error" but really it's an operational error.

People are building multimodal transformers, that try to simulate users.

No matter how stupid the ai, if it can break your ai code, you have a bug.

Tests are just a way of providing evidence that your software does what it's supposed to. If you're not providing evidence, you're just saying "trust me, I'm a programmer."

Think back to grade school math class and your teacher has given you a question about trains with the requirement "show your work." Now, I know a lot of kids will complain about that requirement and just give the answer because "I did it in my head" or something. They fail. Here's the fact: the teacher already knows the trains will meet in Peoria at 12:15. What they're looking for is evidence that you have learned the lesson of how to solve a certain class of problems using the method taught.

If you're a professional software developer, it is often necessary to provide evidence of correctness of your code. In a world where dollars or even human lives are on the line, arrogance is rarely a successful defense in a court of law.

Not quite. Tests are just a way to document your software's behaviour, mostly so that future people (including future you) working with the software know what the software is intended to do – to not leave them to guess based on observation of how undefined behaviour plays out.

That the documentation is self-validating is merely icing on the cake.

That's a beneficial side effect, not the purpose of having tests.

It's like saying the purpose of eating is to fill your senses with wonderful flavours and texture. No, that's a beneficial side effect: the purpose of eating is to prevent death from starvation.

Indeed. You could write the documentation in Word instead, but when the "herbs and spices" come for free...

> If you aren’t perfect at writing tests, how do you know the tests are complete, bug free, and actually useful? :)

I did like the rest of the post, but this is not hyperbole. It's just a disingenuous argument, and one that looks orthogonal to your point that "tests are a tool for a job".

If you aren't perfect at magnetizing iron, and you need a working compass, you better magnetize two needles and use one to test the other. The worse you are at magnetizing iron, the more important it is that you do this if you want to end up with a working compass.

> If you aren't perfect at magnetizing iron, and you need a working compass, you better magnetize two needles and use one to test the other. The worse you are at magnetizing iron, the more important it is that you do this if you want to end up with a working compass.

This is modern testing in a nutshell - it's ineffective but the author of the test can't actually tell that!

Using this analogy, if you created 10 magnetised needles using the wrong process and getting the wrong result, then all 10 would agree with each other and your test passes, but your needle is still broken.

I don't think you understand how magnets work.

Hint: if you think the way to test whether a needle is magnetized using another possibly magnetized needle is by building both needles into two separate compasses, you're nowhere close.

> Hint: if you think the way to test whether a needle is magnetized using another magnetized needle is by building both needles into two separate compasses, you're nowhere close.

I thought it was clear from my post that I do not think this.

I also think you are missing the point.

You wrote:

> if you created 10 magnetised needles using the wrong process and getting the wrong result, then all 10 would agree with each other and your test passes

This suggests that you do think soemthing like this. Again, the way you test wheher you successfully magnetized a needle using another potentially magnetized needle is not by checking whether they "agree with each other" in the end application.

> This suggests that you do think soemthing like this.

Or it suggests they’re continuing the analogy (which isn’t perfect) to make a different point.

> Again, the way you test (…) is not (…)

Twice you’ve spent the majority of words in your post telling someone they’re wrong without explaining the correct methodology. That does not advance the conversation, it’s the equivalent of saying “nuh-uh” and leaving. If you disagree, it’s good form to explain why.

It doesn’t take long to say the failed magnetisation would leave all needles pointing in disparate directions, not the same consistent wrong direction. Unless there’s something else in your test that is so strong and wrong that it causes that problem, in which case the analogy starts working again.

> You wrote:

>> if you created 10 magnetised needles using the wrong process and getting the wrong result, then all 10 would agree with each other and your test passes

You snipped out the qualifier to the paragraph. The qualifier is important. Here's the full quote with the qualifier:

> Using this analogy, if you created 10 magnetised needles using the wrong process and getting the wrong result, then all 10 would agree with each other and your test passes, but your needle is still broken.

IOW, using this analogy for software development, all the products created with the wrong algorithm and/or process would all agree with each other. It's why I say you are missing the point.

You're not creating a binary product that either exists or doesn't exist, like magnetism, so repeating the process as some sort of "test" is a broken way to test.

It's also the most popular way to write tests: the tests are effectively moot because a pass does not indicate that the result is correct.

I don't get this analogy.

Apart from the fact that in your example the produce is validated using the exact same produce, you are actually describing the perfect test:

Two magnetized needles will validate each other, so both the product (needle#1) and the test-setup (needle#2) will be confirmed as valid in one step. If one is not working, the other will self-validate by pointing at the earth magnetic field instead...

The problem with using two needles to test each other (instead of using an external third source like the earth's magnetic field) is North and South could be swapped on both of them. The test would validate correct but the needles be wrong.

I feel like our industry kinda went the wrong way wrt UI frontend tests.

It should be much less focused on unit testing and more about flow and state representation, both of which can only be tested visually. And if a flow or state representation changed, that should equate to a simple warning which automatically approves the new representation as the default.

So a good testing framework would make it trivial to mock the API responses to create such a flow, and then automatically do a visual regression of the process.

Cypress component tests do some of this, but it's still a lackluster developer experience, honestly

This is specifically about UI frontend tests. Code that doesn't end up in the DOM are great for unit tests.