Type checking in Python
senko.net
senko.net
I believe that dynamic typing results from frustration over generics, rather than over static typing in general. (Type stuttering is another problem, but modern languages solve it with local type inference.) I mean, annotating non-generic functions is always straightforward doesn't introduce much complexity:
# s should be str
# returns nothing
def print(s):
...
# just becomes (using Py3 annotation syntax)
def print(s: str) -> None:
...
However, it's much harder to write a type annotation of generic functions. Worse, remember that Python's map is variadic. # for any types t0, t1, t2, ... tn:
# fn is a function that takes arguments of types t0, t1, ... t(n-1)
# and returns a value of type tn;
# seqs are iterables of types t0, t1, ... t(n-1);
# returns a list of type tn.
def map(fn, *seqs):
...
# How to write this signature is non-obvious
Many mature static typing systems would allow you to express such types (usually called parameterized types). But any one of them would require more than those trivial notations in print.That's when the dynamic typing people get annoyed and go "fxxk static typing systems, I can handle this in my mind". Which is about 65% (totally random estimation) the point of dynamic typing in my opinion.
Any type checker for dynamic typed languages that doesn't seriously try to solve the generics problem is not genuinely interesting.
Python 3 annotations are already not used in any specific predetermined way, so if you create a system to do something with type annotations then you can implement List(str) or whatever it is you want an annotation to work like.
True, list comprehension is often used instead. But a complete typing system should be able to type-check that too. The point is not changed.
Also, I only used map and filter as an example. Python programmers tend to write a lot of generics functions without actually knowing it.
def f(g, *args):
...
r = g(*args)
...
return r
This function is actually quite generic, akin to map.And we are not yet getting into the realm of "trait" or "concept" or "typeclass" or whatever.
def sum(*nums):
s = 0
for n in nums:
s += n
return s
Remember, although s is definitely an int, s + n may yield some completely unrelated type if n implements __radd__. The function here involves potentially infinite number of types, and their __add__ or __radd__ traits (reusing the name of the Python magic method). To fully express its type without unnecessary constraints is an interesting exercise. Since Python allows dynamic parameter list building (like sum(*nums)), dependent typing may be necessary for that purpose.Ocaml/SML are both functional languages similar to haskell, and allow IO pretty much anywhere. They don't have typeclasses, (although ML people would say there's nothing you can't do with the ML module system[0]).
uint8_t foo = (uint8_t)69;
In theory, there should be no need to specify the type twice. The ideal circumstance is something like uint8_t foo = 69; List<String> list = new ArrayList<String>();
It feels silly to have to write the type parameter twice. But now you can say: List<String> list = new ArrayList<>();
It's usually not a huge deal, but occasionally the stutter can be more pronounced: Map<String, List<Map<Integer, Set<Float>>>> map = ... map : ('a... -> 'b) -> ('a list)... -> 'b list
Where <type>... denotes <type1> -> <type2> -> ... -> <typeN> spliced into the type signature, with each type variable 'a in <type> replaced with 'a1, 'a2, ... 'aN. You could cover tuples too: zip : ('a list)... -> 'a,,, list
Bit ugly, and writing a type checker for it could be fun, but it seems workable.Right, looks very similar.
However, behold the byzantine Scheme numerical tower and the effort to type it: http://www.ccs.neu.edu/home/stamourv/papers/numeric-tower.pd...
args = []
while some_condition():
args.append('x')
f(*args)
Typed Racket allows you type some common cases, though.In Haskell map is
map :: (a->b) -> [a] -> [b]
How could we do something like that in a Python type extension? One way would be to just use the Haskell notation in quotes: @type("(a->b) -> [a] -> [b]")
def myMap(fn, as):
#...etc...
An alternative would be to define a function Fun for describing functional types, and gen.{something} for a generic type, e.g.: @type(Fun(gen.a, gen.b), [gen.a], ret=[gen.b])
def myMap ...etc...
But that notation doesn't (at least to me) look as clean as the Haskell.Similarly, is there any reason the example `add` function in the OP shouldn't be able to add strings, or lists?
I've seen a lot of these quick-runtime-type-checking libraries, but they always have the same problem as manual type-checking: they make the constraints much stricter than they need to be and prevent entire classes of useful behavior.
The article says that the lack of type checking "lets through" a certain class of errors. However, let's be specific here: there is no build-time static analysis phase in the transformation of python source to executable code. So "lets through" means the same thing whether you have strong typing or not: the error is going to be found at runtime. What we're really talking about, then, is converting AttributeError ("YourObject has no attribute 'startswith'") into something more specific ("Hey, this is supposed to be a string!"). Honestly, that seems to me to be a pretty minor increase in diagnostic information.
So the bottom line for me is that I feel I didn't really understand duck typing. I used to joke that the term actually meant "no typing," and that is in some respects true. But what it really means is "the thing can do what the method expects it to be able to do." If the thing can't serve in the expected role, then the existing errors that result are sufficiently explanatory, imo.
You can have "duck typing" in Haskell, for example, using type-classes.
Instead of "letting the errors through" to run-time, you detect them at compile-time.
Will it fit in with the current Python ecosystem? Will it have to change the way the language is used? Maybe, but that doesn't mean we shouldn't experiment with static analysis. We're hackers, after all, and this is something that probably deserves to be hacked on.
EDIT - looks like obiwan will allow decorators as well as annotations. I do prefer the syntax of typedecorator though. It seems less cluttered.
I would rather see a more generic pre/post-condition contract system, to be used in select top-level functions, that gives a lot more flexibility in expressing what is supposed to happen (i.e. I expect to be given a not-None value with a __str__ method that doesn't throw and will return an iterator yielding consecutive not-empty, not-None strings), including what assumptions you're making when you call a function that you got from a random object (i.e. here I'm calling a function that I got somehow as a parameter and I'm going to assume it can take a string of the format "ipv4 address:port" and returns an object that is a database connection), together with a mechanism for recovering from broken contracts - we are doing runtime checks, so there's no reason to restrict ourselves to Java-style type constraints, which don't even work for Python in general. Or, alternatively, a lighter system that can be checked at import time once before the program is started "for real". Preferably both of those.
The closest I've seen is the traits thing from Enthought, but there doesn't seem to be much buy-in from python users.
Turns out with modern tools you _can_ statically type check python :)
I wish there was something like this, but parsing the docstrings with the same format as pycharm: http://www.jetbrains.com/pycharm/webhelp/type-hinting-in-pyc...
@returns(int)
@params(a=int, b=int)
def add(a, b):
return a + b
becomes: @typ(int, int, ret=int)
def add(a, b):
return a + b
I was inspired to do it like that by Haskell's very clean type syntax.Senko's version has the advantage that you can compose types in it, e.g. {str:int} is a dictionary whose keys are strings and values are integers.
@ensure_annotations
def f(x: int, y: float) -> float:
return x+y
f(1, 2.3)
>>> 3.3
f(1, 2)
>>> ensure.EnsureError: Argument y to <function f at 0x109b7c710> does not match annotation type <class 'float'>
I think it works better than other approaches mentioned here because it (1) doesn't repeat the contents of the function signature, (2) doesn't install any magic system-wide hooks (and the attendant performance issues), and (3) is completely optional.When function annotations were introduced, I ended up removing all their uses from the standard library because every early attempt to use it was too simplistic and failed to support any type-system use case except for extra documentation.
"The module Data.Dynamic uses Typeable for an implementation of dynamics."
http://hackage.haskell.org/package/base-4.7.0.0/docs/Data-Ty...
Or more specifically, how non-intuitive it is to use the IO stuff
I don't care what it uses, or what is it called, I care about being able to use it with what I know
So yeah, I'll go for Go instead of Haskell
It is unfamiliar, but if you use static typing, at least reap the benefits of better error checking, no runtime null dereferences, etc.
That sounds a pretty good description of Go, so I'm wondering what his point is in the first place.
I think when the first dynamic programing language was invented, the loose typing system must be thought as a big advantage.
Type checking is necessary in some cases, but I really enjoy dynamic typing a lot. So I think a bit tradeoff like this is acceptable. After all, we have to tradeoff everywhere when it comes to computer science such as the time-space tradeoff of an algorithm.
Once you use a good static type system (ML, OCaml, Haskell) you see that dynamic typing isn't responsible for the joy, but the ability to express rich ideas without repeating redundant type declarations.