*in Javascript.
In most other languages where unwanted extra arguments raise an error instead of being silently ignored, this mostly isn't a problem
In most other languages where unwanted extra arguments raise an error instead of being silently ignored, this mostly isn't a problem
The true nature of type checking is basically a method of hindering you. The set of all correct programs is much smaller then the set of all programs that exist so anything that hinders a programmer from operating in the bigger parent set outside the set of correct programs is a good and impressive thing.
What’s going on here is a type checking issue. JavaScript and typescript is a little too loose. The map method takes a function of <Arity 3, 2, or 1> So if a library changes a function from <arity 1> to <arity 2 or 1> you should get a type error, but the type checker is too loose. It’s subtle.
Basically a type of <arity 3, 2, or 1> should only type check with <arity 3> or <arity 2> or <arity 1> it should not allow <arity 2 or 1>. You see what’s going on here? Subtle.
This is indeed as much of a type checker problem as it is defining what type correctness is. The definition above is simply a way of defining type correctness that fits with our intuition of what is correct for this given situation, so take what I wrote with a grain of salt. There could be situations where the current definition of type correctness in typescript is more correct then the definition I provided.
Our intuition is complex and if you think long and hard enough you may be able to come up with a formal definition of type correctness that perfectly fits our intuition and therefore elegantly unionized typescripts looser definition of correctness and my own stricter definition.
Beware though, often human intuition can be contradictory. This means that a formalization of our intuitive notions of type correctness will also be contradictory and therefore unusable. In other words there may not be a way to type check for this issue while maintaining the convenience of the status quo.
Intuitively I think it’s possible, you just need special syntax to tell the type checker whether to use my stricter definition or the original looser definition that’s in use now.
Also I’m not sure if there’s any type checker in existence that handles that case (don’t know). So I believe this is more than just a JavaScript issue.
It is my very strong suspicion that in almost all (if not all) of the similar cases in programming methodology, there have been not just arguments on both sides, but implementations and real-world lessons on both sides.
A veeeery minor example: I’m equally as sure that someone designed and deployed systems that specifically created errors when you tried to pass less than the required number of params as I am that other people (or even the same people) specifically designed and implemented systems where params were optional (most likely because they hated having to specify empty params every time).
Sometimes a dynamically typed language is the right tool for a job, but IMHO this mostly holds for auxiliary stuff. Typed languages are more effort during programming/learning, but the benefit is gigantic.
But maybe I'm just too biased because I'm usually involved with "is this fails, people may [literally] die".
This is really just a wonderful example how javascript is a mental burden to the programmers instead of a useful tool.
TypeScript won't catch the original landmine because it ignores the extra parameters; maybe some linter would? Is there a rule that enforces "functions used by map must spell out all the parameters?"
* Example of typescript giving an error on toReadableNumber_v2: https://www.typescriptlang.org/play?#code/FAYw9gdgzgLgBAQwE5...
So username, password are distinct from string and objectid might be backed by an int, but is distinct from int.
Obviously, once compiled, there's no overhead, it's a zero cost abstraction. But an incredibly useful one.
Scenario: Library changes signature so you always pass functions with more parameters.
Result: This will break almost every codebase, they probably wouldn't do this in a minor patch.
Scenario: Library adds a new signature where you can pass functions with more parameters and keep them overloaded between each other.
Result: Compiler can't identify which of the two signatures to pass your overloaded function and throws a compilation error.
A type check could prevent this because it would require map to take a reference to a function with three parameters, or the compiler would complain inside the map implementation.
Similarly, passing it a function with only one parameter would be a type violation and the compiler would complain.
Now in a language with type checking, you could still potentially run afoul.
Say the map function was overloaded with one variant for one-parameter callbacks, one variant for two-parameter callbacks etc. Then the compiler might figure out it could use the second overload if the "toReadableNumber" function got changed to take the extra "base" parameter.
So again you end up with the numbers getting converted with a variable base.
Though, IMHO, having such an overloaded map function is inviting trouble and is a very poor design.
[1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Functions can be assigned at runtime. It would have to be a runtime error.
So it's NOT a runtime error, it's a compile-time error, even though you can assign different functions at runtime.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Static code will get checked at compile-time, even though the exact function passed as callback depends on the incoming data via a series of if/case statements say, from a dictionary or whatever.
And it only works because `Array.prototype.map` callback has one required arguments and two optional ones. What language with optional function arguments protects you from this sort of behavior? Do people in that language not test this sort of behavior? Or at least run through the app?
More than that, the whole hurrdurr-javascript-bad thing is so tired. Lots of work is being done in JavaScript. Sure, it has its quirks, but those quirks come with expressiveness. You can use functional or imperative style, throw lambdas around, and it runs on pretty much all the phones and computers in the world.
Languages with a sane type system.
TypeScript has no issues catching such mistakes. Writing tests to catch simple type errors is such an incredible waste of time.
TypeScript doesn't complain when you pass in a function that ignores some of its arguments. Which is totally fine and safe. If you upgrade your function from no second argument to a numeric second argument TypeScript will not complain and your program might break.
It will not crash, because it still perfectly type-safe, but it might not behave like you want to. So in that sense the article has a point.
However, this is just one instance of a larger issue with changing the behavior of a function, while keeping the types compatible.
- Using a function as a callback to .map() with two numeric arguments and then swapping the arguments.
- Returning a tuple of two of the same type and swapping the order.
- Returning a string in a new encoding.
- ...
Which is generally worse than crashing, because silent data corruption can have far-reaching impacts.
> If you upgrade your function from no second argument to a numeric second argument [...] It will not crash, because it still perfectly type-safe
It's type-safe using a definition of "type-safe" that is defined relative to the underlying JavaScript model of "corrupt the user's data rather than crash". It wouldn't be type-safe using most other languages' (including Python, for example) model of "functions have a type that includes the number of arguments, and applying them to the wrong number of arguments is meaningless and hence not considered type-safe".
It's fair for TypeScript to use this approach. But it is surprising to many of us, who view type systems as tools for ruling out some dumb functional bugs, not just crashes.
Also, > A lot of work is being done in JavaScript
Is that it? It's one of the best funded languages on Earth, what do you expect - with D for example we can do all the things you mentioned and catch errors like this, and we're basically just some guys working on the language not the combined might of the entire Internet sector.
Array.prototype.map = function (func) {
if (func.length !== 3) {
throw new Error(
"bad! this function you've passed must take THREE arguments. Grr!"
);
}
for (let i = 0; i < this.length; i++) {
this[i] = func(this[i], i, this);
}
return this;
};
(making this workable and bug free is left to the reader or multi-billion dollar corporations)
There can be a non-indexed map as well.
I'm most curious as to who uses the last argument in the JS map function!
array.map((val, index, {length}) => if (index + 1 == length) { alert("last element") } else { alert("element #"+index) })
It's convenient for some things that would otherwise require a reduce (but where reduce isn't particularly more efficient, because you just need lookahead/lookbehind) or an imperative loop, like transforming a list to a set of moving averages over the list.
It's a little more expressive than reduce our imperative lots loops in those cases, too.
const func = (i: number, x: boolean) => i * 2
const nope = [1, 2, 3].map(func) // type error!Unfortunately, this does mean TypeScript’s type system can’t be entirely sound. A classic situation that is also legal according to the rules of TS but “ought” to fail type checking is something like this:
let arr_num: Array<number> = [1, 2, 3]
let arr_opt: Array<number | null> = arr_num // Erm...
arr_opt[0] = null // ERM!!!
This problem could be fixed by making it an error to alias arr_opt to arr_num. However, that might also cause a lot of extra work for anyone trying to migrate an existing JS code base, particularly if the types involved are not of their choosing but instead determined by code written elsewhere.
For example, if you called a library function that returned an Array<number> and you passed that into another library function that required an Array<number | null> and wasn’t going to modify that array, enforcing the constraint could mean that working code was broken for no real benefit.
Then you get into deeper questions about enforcing immutability using the type system, and finding that again you’re building on sand because you still have JS underneath. IMHO, it’s hard to blame the TS designers for not wanting to go down these kinds of rabbit holes.
However, you don’t see the same warning in the case of functions that can be called with variable numbers of arguments if the types of the arguments being unintentionally supplied do match, because within the rules of TS, this is working as designed.
Combined with the perhaps unfortunate decision to provide a standard `map` function that doesn’t use its callback as most languages do, there is still the potential for an unexpected change of behaviour that the type checker can’t warn you about here.
Yes, TS is fine with passing more arguments to a callback that takes fewer. The callback cannot possibly use the additional arguments, so it doesn't matter what gets passed as it will not change the outcome.
This is very different from passing the wrong kinds of arguments to functions that do read them and do something with them, like parseInt.
Now, if you decide to pass a function with an optional second argument that matches the second argument that will get passed to the callback and expect that it will not be used because why would anyone pass additional arguments to a map callback - then yes, you will have the problem again.
function addOneByDefault(num: number, addAmount = 1) {
return num + addAmount
}
[1,2,3,4].map(addOneByDefault) // this typechecks but works poorly
The type checking I am talking about is not a sum type. It is not that the function can take a two different possible types. It's the fact that the parameter function can mutate into two different types depending on the usage. It has (<arity 1 or 2>) not (<arity 1> or <arity 2>) if you catch my meaning.... Or in other words the concrete type is not evaluated when you pass the function as a parameter but only when it is called with a certain amount of parameters... which is not something type checkers I know about look for.
Perhaps I’m not correctly understanding your idea around arity as part of the function types, but so far it’s not obvious to me how what I think you’re describing helps to resolve that contradiction. Are you suggesting a way the type system could be changed without causing those additional, unwanted side effects?
Do you by any chance have a more rigorous definition or even a formal semantics for your proposed arity types that you could share, so the rest of us can understand exactly what you’re proposing here?
You don't need to change the behavior of the program. You can change the type checker to catch the unwanted error.
>Perhaps I’m not correctly understanding your idea around arity as part of the function types, but so far it’s not obvious to me how what I think you’re describing helps to resolve that contradiction. Are you suggesting a way the type system could be changed without causing those additional, unwanted side effects?
It's not formalized anywhere to my knowledge and I'm not willing to go through the rigor to do this in the comments. But it can easily be explained.
Simply put, what is the type signature of a function that can accept either two variables or one variable? I've never seen this specified in any formal language.
To fix this specific issue you want the type signature here to specify only certain functions with a fixed arity.
When some external library is updated with a function that previously had arity 1 to <arity 1 or 2> that could be thought of as type change that should trigger a type error.
Right now type checker recognizes F(a) and F(a, b=c) (where c is a default parameter that can be optionally overridden) as functions with matching types.
F(a) == F(a, b=c)
F(a,b) == F(a, b=c) <-----(F(a,b) in this case is a function where b is NOT optional)
F(a) != F(a, b)
This is exactly the problem the author is describing. The type checker assumes that when the library changed F(a) to F(a, b=c) that the types are still equivalent, but this breaks transitivity so it's a bad choice and will lead to strange errors because programmers assume transitivity is a given.
You don't see this problem in other type checkers because JavaScript is weird in the sense that you can call a function of arity 1 with 5 parameters.
While blocks, procs, and lambdas all have arity metadata, only lambdas check for the argument count when called. The other two drop excess arguments and fill missing arguments with nil.
If you're inlining your block as a literal do-end block on the call site, it's just a matter of knowing what kind of data you're calling the block-taking method on. So blocks are kinda different.
If you're designing a more intricate piece of code to be used repeatedly by a 'map' or 'reduce' (like in the example), nothing is preventing you from defining a lambda instead of proc. And nothing is preventing you from designing your library so that it exposes only arity-checking lambdas to the outside.
But it's also quite usual to define callbacks as plain old methods (e.g. Rails before and after actions). Methods can be easily used as a block by getting the actual Method object first with the 'method' method, then using the & syntax to automatically convert them to a proc (e.g. map(&method(:foobar)) which again, converts them to arity-checking lambdas.
As you mentioned, lambdas and methods check for that, but it’s sad to have to give up the syntactic and lexical niceties of blocks.
function toReadableNumber(num, base, trap) {
if(base == undefined) base = 10;
if(typeof base != "number") throw new Error("Second argument should be the base! base=" + base + " (" + (typeof base) + ")");
if(trap != undefined) throw new Error("Did not expect a third argument. Are you using this with map? Then use an intermediate function.");This is a problem that should be handled at the language level, not by adding multiple lines of potentially incorrect code for every dozen lines of regular code.
Also you should wait until your API is somewhat stable before adding the guards. So for most code, you do not need guards. But if your code is used by many, that defensive coding/guards, taking only a few minutes to add, will save countless man-hours that would otherwise be spent debugging.
As a general rule I like errors to throw early. So when I found a bug, (I first write a test to automatically reproduce the bug, then) I backtrack and add guards to each step (with helpful debug/data in the error message), so that the bug would be caught at the surface, rather then causing weird issues several layers down.
And guards are much easier to write then complicated type definitions. And the errors will be more informative, helpful and human friendly then errors from a type-checker.
Defensive coding is mostly useful in long living apps that have a lot of state, and which is constantly developed (new features added, breaking changes, etc). You would not need defensive code in programs that are executed once and then thrown away.
Most definitely not. It's not allowed in Python, it's not allowed in Ruby, it's not allowed in any Lisp I know of[0], … it is allowed in PHP, which is about what I'd expect from that[1]. In most dynamic languages the arity is not a suggestion[2].
Which is exactly the issue at hand: `Array#map` was (stupidly) defined as calling its callback with 3 parameters. The last 2 are useless 99.99% of the time (and in better language you'd compose them in if and only if you needed them), as a result it's almost universal that you'd pass single-parameter callbacks which works… until it doesn't because the callback now takes 2+ parameters and starts taking in account the previously ignored garbage `Array#map` feeds it. The average JS developer likely doesn't even know Array#map callbacks receive 3 parameters, and usually aren't going to think about it: in 99% of cases it's has no relevance whatsoever.
[0] but most lisps make significant uses of variable-arity functions, which is a very different and much more formal proposition
[1] PHP's one saving grace being that HoFs have historically not been much of a thing, though I have not tracked how it's used these days
[2] as long as it's present at all AFAIK in Perl functions don't have formal parameters lists
def hello(a, b = 'world'):
print(a, b)
hello('hello')
hello('hello', 'world')
hello('hello', 'world', 'there)
# nor
def hello(a):
...
hello('hello', 'world')In python those functions would look like:
def F(a = None, b = None, *args): ...