Automatically Make Unit Tests
wiki.call-cc.org
wiki.call-cc.org
The essential complexity of writing unit tests is in formulating your assumptions ahead of time. As you correctly point out, no tool can ever solve this. However, an accidental complexity is correctly expressing your assumptions in code. Tools such as a REPL and this library can definitely help with that.
To be honest, my development REPL logs have a lot of junk, so it would be extra work to write a clean one, and then make the right assertion (maybe I was looking for "even number" not "exactly 132"). Lisp-y REPLs are friendlier than some, but it would be cool to associate it with something like a notebook where it is easier to update imports/mocks/etc out of order. Cool idea though.
1. Unit testing is a kind of functional testing. On a unit.
2. And regression testing is rerunning your existing functional tests after, say, refactoring, to see if they pass.
Complex? Confusing? Which part of this precisely is confusing?
The headline says they're generating unit tests automatically.
Which means I expect that I run a command, and it plops out unit test code. automatically. without my interaction.
The article is about recording repl interactions. that's not automatic.
and really, they look like implementation details? so bad examples? I don't know scheme, so shrug
You gotta pick a better hobby.
So if I say 'I wrote a unit test' what kind of test do you expect?
In general, I prefer to start writing tests, just enough the tests fail but call the functionality (this makes API design flaws obvious), then actually build the functionality and, finally, make the test conditions reflect what we need.
Now, bear in mind that I say this as a passionate tester and verification enthusiast. I don’t have a better alternative, so I keep writing tests. And yes, they do catch bugs. I’m not saying they have no value. I’m saying, they always let some bugs through, I.e, they are not complete.
And that makes the cost of them very depressing. Not only cost to write, cost to continuously run.
REPL-driven development is basically test-driven development with sorter feedback cycles during active development, but the downside is you need to make an extra effort to go back and preserve the important tests for posterity. This might well reduce some of that friction.
That said, I suspect I'd find a similar limitation with this as with Hypothesis. Most of the tests I write aren't for pure functions, or require a fair amount of setup or complex assertions. It's possible to write test helpers that do all of this, and I do, but too much of that means tests that are complex enough to need their own tests, so I think it's important to balance.
This is probably specific to my normal work (Python, web services), but I'd suspect applies to a lot of testing done elsewhere.
In response to this, I recommend the "Functional Core, Imperative Shell"[0] talk/pattern. The idea is to extract your business logic as pure functions independent of the data access layer. This pattern allowed me to test large portions of a code base using property tests. This works really well in most cases and gives me much more confidence in the product that will be deployed.
[0]: https://www.destroyallsoftware.com/screencasts/catalog/funct...
While the pattern is good to always be striving for, I think it can be too much boilerplate at times.
If I've got say 100 lines of business logic, it makes sense to have 10 lines to do the necessary data shuffling to get everything in order so that you can keep your business logic a nice pure function.
On the other hand, if you have 5 lines of business logic, it maybe doesn't make sense to have 10 lines of shuffling data to be able to keep the business logic a pure function. Not so much for the time to write it but because it can obscure the functionality for the reader.
> Wouldn't it be cool if we could have code that would test itself?
> Yeah, if only there were some kind of automated way to encapsulate the requirements of classes and methods that could ensure that they behaved within some known boundaries...?
We've had automated unit tests for quite some time, but they are more often called compiler checks. Want your "automated unit tests" to be more comprehensive? Improve your type system and your compiler!
With all that said, I do not want to be dismissive of peoples' projects. This is fun and neat.
Joe Armstrong(of the Erlang fame), talks about this a bit here: https://youtu.be/TTM_b7EJg5E?t=778
I've pointed to a specific timestamp, but there might be more details somewhere else in that talk.
These are based on powerful types as "dependent types" which can encode mathematics and programs in the same programming language, and allow the mathematics to "talk about" the program. The mathematical underpinning is the Curry-Howard-Lambek correspondence.
[1] https://coq.inria.fr/ [2] https://github.com/mit-plv/fiat-crypto [3] https://compcert.org/ [4] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
Too many people looking for silver bullets.
I would guess that Jupyter notebooks could easily be made into such a thing as well. Also doctests with editor integration might work like this already?