What sort of tests don't you end up writing? I ask because I work in both dynamic and static languages, and find myself writing very similiar tests in each case.
For the tests the check types of values, that's just a dumb test that you shouldn't write regardless of the language.
Then for rest of the post, he talks about having nicely-defined tests which is orthogonal to whether or not your types are static.
The thing is, almost all the tests you'd write in a dynamic language end up being something you can encode into the types of values.
Can you actually give an example of a test that would be realistically written in a dynamic language that would be something encoded into the types of values?
There is a definition for the schema to a fruit table.
Fruit: {
list_id: 32
}
The actual options are populated from a list:
const LIST_OPTIONS = [
[32, "Blueberry", 1],
[32, "Apple", 2],
[32, "Orange", 45],
[45, "Nope", 1]
]
And for the actual test:
This actual sends the request to be processed.
let response = await processRequest({
type: 'QUERY',
table: 'Fruit',
})
deepStrictEqual(response, {
status: 'OK',
records: [{
id: "Blueberry",
name: "Blueberry",
position: 1
}, {
id: "Apple",
name: "Apple",
position: 2
}, {
id: "Orange",
name: "Orange",
position: 45
}]
That aside, what pieces could actually go wrong in a statically typed language? You can't typo field names or the like. You could typo the names, but I don't believe typing them into the computer twice adds value - better to check them once in code review. People aren't going to be editing this code in a way that could lead to getting the names wrong.
You couldn't go down the wrong code path and end up with the "normal table" query, because having list_id instead of something else is part of the type. You couldn't pass some other number as the list id, because list ID is a typed thing that is different from all other IDs, and in any case where would an outside number come from? And you're definitely going to return a response that's formatted with a record list, because that's part of the type of the method. (You might need to test the JSON <-> String piece, but that's generic, so you only need an O(1) test for that, you don't have to test every JSON endpoint. In practice you're likely using an existing well-tested library. Similar reasoning applies to the database querying part). Assuming you have warn-for-unused-parameters enabled, you couldn't forget to apply the criterion and just return everything in the list.
So yeah, assuming this was for part of an existing system where I'd already tested generic facilities like database access and JSON serialization, I wouldn't feel the need to write this test at all.
But this is the generic facility for database access. This is just an additional special case it has to handle. Does that change your perspective on whether you would write a test for it?
> That aside, what pieces could actually go wrong in a statically typed language?
Well, I could type the SQL wrong, but let's assume we are working in a sufficiently awesome statically typed language that would catch that kind of error.
I could filter on the wrong column, with the wrong operator, with the wrong value, on the wrong table. I could decode the name column using the wrong encoding. I could return multiple rows per option. I could do a huge number of other things that I couldn't think of. These may not be really likely to be mistakes I'd make (this is a pretty simple task), but several refactorings down the line, I can easily see an issue being introduced.
At the end of the day, even in a statically typed language where much of the functionality would be verified by the compiler, I'd still like to have a test on this.
> Ok, good example on some levels, though it's doing a lot of interfacing with untyped parts
Hmm... the fact that it deals with the boundary of the system and the fact that it deals only with converting data from one format to another might render it a poor sample (i.e. there's not really much interesting logic). I can come up with another if you are interested.
Presumably it would be using a lower layer - or more likely an existing library - for doing the actual query though?
> I could filter on the wrong column,
Not if you've got a typed representation of which column's which
> with the wrong operator,
Not if equality is the only operator available. This is one of my biggest reasons for using wrapper types around IDs - you shouldn't treat IDs as integers, because the idea of adding or multiplying or <=ing an ID makes no sense.
> with the wrong value,
How? You have to use the passed ID for something, and it can't do anything else.
> on the wrong table.
So make the table part of the type. A list ID is a different thing from a user ID, they shouldn't be compatible.
> I could decode the name column using the wrong encoding.
Encoding should absolutely be handled with types.
> I could return multiple rows per option.
Hmm, maybe. But I can't see how you'd do that by accident.
> At the end of the day, even in a statically typed language where much of the functionality would be verified by the compiler, I'd still like to have a test on this.
I started out thinking the same thing (I moved from Java/Python to Scala). I've gradually reduced the amount of tests I write over four years as I've found them to not add value. I was as surprised as anyone, and even privately entertained the thought that maybe I was some kind of super-programmer who was smarter than all those people who needed tests, but that month or two of trying to do stuff in Django made it very clear that it wasn't me, it was the language.
> Hmm... the fact that it deals with the boundary of the system and the fact that it deals only with converting data from one format to another might render it a poor sample (i.e. there's not really much interesting logic). I can come up with another if you are interested.
Yeah, happy to if you're interested. It's a useful exercise in thinking it through for myself as much as anything.
Ah, Scala. I could be prepared to believe that this is true for Scala (and similarly advanced typed systems). It is when people claim they write fewer tests in java than in python that I found it really dubious.
> Yeah, happy to if you're interested. It's a useful exercise in thinking it through for myself as much as anything.
Ok here's another test:
These are action objects, they represent a sequence of user actions on a JobExtra.
const actions = [
JobExtra.actions.load(undefined),
JobExtra.actions.field('changeAmount', Money.actions.set(new Decimal("12.45"))),
JobExtra.actions.field('certifiedForemanRelated', Checkbox.actions.set(true)),
JobExtra.actions.field('appliedPercentage', Percentage.actions.set(new Decimal("4.5")))
]
const state = _.reduce(actions, (state, action) => JobExtra.reducer(state, action), undefined)
strictEqual(true, isJobExtraValid(state))
1. The changeAmount must be set to non-zero 2. The certifiedFormanRelated must be false or the appliedPercentage must be true
But, if the user modifies the appliedPercentage, the certifiedFormanRelated should be automatically set to true.
So this, and a number of related tests verify that under different sequences of actions I end up the result I expected.
Is there a way to push this into the type system?
For 1., you could have a non-zero number type, or you could settle for a check in the JobExtra constructor. For 2., it sounds like this is a coproduct situation - a JobExtra is either certifiedFormanRelated or has an appliedPercentage. Scala doesn't quite have native support for coproducts the way it does for tuples, but you could represent it with a sealed class hierarchy:
sealed trait ForemanOrPercentage
case class ForemanRelated() extends ForemanOrPercentage
case class AppliedPercentage(value: Decimal) extends ForemanOrPercentage
In Java there isn't a "sealed" keyword, but you can emulate it with the visitor pattern - you'd define a ForemanOrPercentageVisitor and only ever operate on ForemanOrPercentages via the visitor. That way if someone creates their own subtype of ForemanOrPercentage it still has to behave like one or the other, because they still have to implement visit().
Since the user input modifies this object, the user can put the object into an invalid state. In fact, when the object is first created it is in an invalid state. So I can't simply define my types so that invalid state is impossible.
That's very interesting, and shows how types can go much further then what we see in languages like Java. I'll have to try out one of these languages some day and see how well it works in practice.
No, I don't think so. I'll write a test that uses the library. If the library returns null when I don't expect it, it'll go into my code an cause an exception to tbe thrown. I'm not deliberately testing for nulls, I'm testing generally that my code does what I think it does.
The kind of library function that returns null doesn't usually return it on the "happy path", it usually returns null for some weird condition you haven't thought of. I mean, if it's a function that returns null often or that you'd expect to sometimes return null, you'll wrap up the return value in an option straight away even without having written a test. The kind of library function that causes you to get exceptions in production is the kind that returns null in some weird corner case - but unless you think of the weird corner case, you wouldn't hit it in a test either.
If I was using a library with some complex stateful interface I might write tests for the interaction with that (I'd probably put a facade on it, keeping it separate from the business logic, and the facade would have unit tests) - but I'd try to avoid using such a library. And I'll always have a few high-level end-to-end tests (not really "unit" tests) as a basic "does it work at all", because that kind of mistake is easy to make. But the bread-and-butter unit tests that I wrote when I worked in Python just don't seem valuable or necessary.
Honestly, I can't say how it would work if I was using a java code base that consistently applied these kinds of ideas. You may be right, but at this point I have hard time believing.
Depending on where you are in the security vs efficiency spectrum, you may opt to ignore those assertions in production.
Writing specific tests for that looks very inefficient to me but maybe I am missing something.
If some function A calls function B with a wrong type, the problem is with function A. So basically, there should be only type tests after IO functions (probably after parsing), the rest can be assertions.
At some point you have to assume something is there and correct (e.g., you don't write tests to check whether the tests are there, also you assume the testing tools work, ...).
In static languages you also assume the programmer set the right type for the function argument, so you might as well assume to have an assertion in the critical code of the functional languages.
If you excel at compositional design then this may work OK but a lot of code bases I have worked on just need a good ol' refactor to sort the mess out. And then the "there and correct" part could fail due to human error, or the new person not knowing the old assumptions etc. The asserts are a step in the right direction in documenting the assumptions though. Types go one level better by both documenting the interface, and enforcing that contract at compile time.
* true as in all possible inputs and outputs rather than merely all code paths
Your comment just solved the problem by ignoring the latter, not much of a contribution to say the least.
I agree that writing specific tests for that is very inefficient, and that few people (myself included) actually do it. I just also think the fact that people don't really write tests for a (anecdotally) common cause of bugs, and instead wait to see them in their production logs, is an anti-pattern, rather than a positive thing.
I want the function to fail when it is passed the wrong arguments, but I don't really care about how exactly it fails. If the wrong types get passed to a function, I'm already screwed and there's no way to do the right thing at that point. I only care that I get enough information to debug what happened.
In pretty much any other case, I want to function return some precise value or do some precise action. These are situations that the code will encounter during normal execution, and so I do care about what exactly it does.