Comment by Jeaye | Hacker News Reader

Comment by Jeaye | Hacker News Reader

I strongly recommend that anyone using spec for validation should check out Orchestra, to instrument the functions and have them automatically validated on each and every call: https://github.com/jeaye/orchestra

For my team, generators and parsing are basically useless with spec. We just don't use them. But describing the shape of data and instrumenting our functions, using defn-spec, to ensure that the data is correct as it flows through the system is exactly what we want and nothing I've seen in Clojure land does it like spec + Orchestra can.

I think part of this may boil down to different types of testing. We primarily use functional testing, especially for our back-end, so we're starting from HTTP and hitting each endpoint as the client would. Then we ensure that the response is correct and any effects we wanted to happen did happen. This is much closer to running production code, but we do it with full instrumentation. Being able to see an error describing exactly how the data is malformed, which function was called, and what the callstack was is such a relief in Clojure.

    Call to #'com.okletsplay.back-end.challenge.lol.util/provider-url did not conform to spec.

    -- Spec failed --------------------

    Function arguments

      (nil)
       ^^^

    should satisfy

      (->
       com.okletsplay.common.transit.game.lol/region->info
       keys
       set)

    -- Relevant specs -------

    :com.okletsplay.common.transit.game.lol/region:
      (clojure.core/->
       com.okletsplay.common.transit.game.lol/region->info
       clojure.core/keys
       clojure.core/set)

    -------------------------
    Detected 1 error

I'm onboard with orchestra because I tend to be lazy. But I also want to explain the rational for why Spec's instrumentation only instrument the input.

This has to do with the philosophy. If you want to write bug free programs, and I mean, if you care A LOT about software correctness.

The idea in that case will be that all your functions will have a set of unit tests and generative tests over them that asserts that for most possible inputs they return the correct output.

Once you know that, you know that if provided valid input, your functions are going to return valid output. Because you know your function works without any bugs.

Thus, you no longer need to validate the output, only the input. Because as I just said, you now know that any valid input will result in your code returning valid output as well. So re-validating the output would be redundant.

And this goes one further. After you've thoroughly tested each functions, now you want to test the integration of your functions together. So you'd instrument your app, and now you'd write a bunch of integration tests (some maybe even using generative testing), to make sure that all possible input from the user (or external systems if intended for machine use) will result in correct program behavior and an arrangement of functions that all call each other with valid input.

Once you've tested that, you now also know that the interaction/integration of all your functions work.

At this point you are confident that given any valid user input, your program will behave as expected in output and side-effect.

You can thus now disable instrumentation.

But before you go to prod, you need one more thing, you have to protect yourself against invalid user input, because you haven't tested that and don't know how your program would behave for it. Thus with Spec, you add explicit validation over your user input which reject at the boundary the input from the user of invalid.

You now know there things:

1. All your individual functions given valid input behave as expected in output and side-effect.

2. Your integration of those functions into a program works for all given valid user input.

3. Your program rejects all invalid user input, and will only process valid user input.

Thus you can go to prod with high confidence that everything will work without any defect.

---

Now back to orchestra. Orchestra assumes that you weren't as vigilant as I just described, and that you might have not tested each and every function, or that you only wrote a small amount of tests for them which only tested a small range of inputs. Thus it assumed because of that, probably when you go towards running functional/integ tests, you want to continue to assert the output of each function is still valid, as you anticipate those will probably create inputs to functions that your tests over that function did not test.

Something like Haskell, or even Rust, requires similar vigilance in order to get the program even into a working state. With thorough, strong, static type checkers, novel borrow checkers, and more, a lot of development time is spent up front, dealing with compiler/type errors. Thus you can go to prod with high confidence that everything will work without any defect.

Now, back to Clojure. Clojure assumes that you weren't as vigilant as I just described, and that you don't have static type checking for each function, or that you don't have a fixed domain for all of your enums. Thus it is assumed because of that, probably when you go running toward testing (unit, functional, or otherwise), you want to assert the validity of all of this data.

My point in re-painting your words is that we all trade certain guarantees in correctness for ease of development, maintainability, or whatever other reasons. Developers may choose Clojure over Haskell, for example, because maintaining all of that extra vigilance is undesirable overhead. Similarly, developers may reasonably choose not to unit test every single function in the code base, but instead functionally test the public endpoints and unit test only certain systems (such as the one which validates input for the public endpoints), because maintaining all of that extra vigilance is undesirable overhead.

Also note that if you try thinking with types, you may start seeing them as tools rather than overhead.

A good blog post about this is:

https://lexi-lambda.github.io/blog/2020/08/13/types-as-axiom...

Reply on news.ycombinator.com