Comment by deathanatos | Hacker News Reader

Comment by deathanatos | Hacker News Reader

Parent Full thread

> You start writing assert fibonacci(15) == ... and already you’re forced to think. What does fibonacci(15) equal? If you already know, terrific—but what are you meant to do if you don’t?

Um …duh? Get out a calculator. Consult a reference, etc. Otherwise compute the result, and ensure you've done that correctly, ideally as independent of the code under test as possible. A lot of even mathematical stuff has "test vectors"; e.g., the SHA algorithms.

> Here’s how you’d do it with an expect test:

  printf "%d" (fibonacci 15);
  [%expect {||}]

> The %expect block starts out blank precisely because you don’t know what to expect. You let the computer figure it out for you. In our setup, you don’t just get a build failure telling you that you want 610 instead of a blank string. You get a diff showing you the exact change you’d need to make to your file to make this test pass; and with a keybinding you can “accept” that diff. The Emacs buffer you’re in will literally be overwritten in place with the new contents:

…you're kidding me. This is "fix the current state of the function — whether correct or not — as the expected output."

Yeah… no kidding that's easier.

We gloss over errors — "some things just looked incorrect" — well, but how do you know that any differently than fib(10)?

It is called snapshot testing, very valid technique. Maybe not best suited to a mathematical function like they have here, but I have found it useful for stuff like compilers asserting on the AST, where it would be a pain to write out and assert on the output and may also change shape.

TIL. That looks like a nice way to add tests to legacy code without having to re-create what TDD would have had the developers started that way.

It is indeed a good way to add regression testing to code with no tests. But it's no substitute for TDD. It can't tell you why something is the way it is, nor can it distinguish between intentional and incidental (although maybe some would argue you shouldn't, given Hyrum's law and all). But it will at least guide you as you try to figure that out and stop you breaking stuff constantly.

The problem is some stuff is a real pain to test with static assertions. Such as I was saying about compilers. It would be a real pain to maintain an expected AST in a unit test, then you'd have to go rework it all if you change the shape and or add/remove nodes etc.

You can mix the approaches, have some static assertions(as sanity checks) but make most snapshot tests. Like I said I wouldn't use snapshot testing for a fibonacci method, but there are problems out there that are a real pain to test via static assertions.

A lot of tests are designed as regression prevention. You know the system is working as designed, but what if somebody comes along and changes the Fibonacci function to compute much more efficiently (and, in the process, makes some arithmetic errors?).

I think “test the function does what it does” is not necessarily the intent here, it’s being able to write tests that fill themselves in and assuming you’ll double check afterwards.

That said, I don’t see how it’s much different to TDD (write the test to fail, write the code to pass the test) aside from automating adding the expected test output.

So I guess it’s TDD that centres the code, not the test…

Yeah this is one of the weirdest takes ever.

"If you already know, terrific—but what are you meant to do if you don’t?"

You're supposed to look at the first gif that visualizes a waveform diagram. How are HDL designs tested? Testbenches (akin to unit tests) and model checking. With model checking you define the property you want to test and the model checker will try to find a counter example.

Said property is so obvious for fibonacci, that it is staring right at your face and you're consciously trying to avoid looking it in the eyes. Fibonacci is defined as fib(n) = fib(n-1) + fib(n-2), so that's what you need to test. This means you can simply test fib(1) = 1, fib(2) = 1, fib(3) = 2, for a fixed set of n to cover the edge cases, then choose a fixed set of random n and make sure that fib(n) = fib(n-1) + fib(n-2) is true. Obviously the only way to be 100% sure is to use a model checker and write code that is bounded in its runtime.

> Um …duh? Get out a calculator. Consult a reference, etc. Otherwise compute the result

Article:

> This is a perfectly lovely test. But think: everything in those describe blocks had to be written by hand. The programmer first had to decide what properties they cared about... then also had to say explicitly what state they expected each field to be in. Then they had to type it all out.

The article is about not getting out the calculator.

> The programmer first had to decide what properties they cared about... then also had to say explicitly what state they expected each field to be in.

Yes, this is the point of testing. You have to think about what you're about to write! Before you write it! The technique in the article completely discards this. It's a terrible way to write tests.

Reply on news.ycombinator.com