The state of type hints in Python

It's the bash/perl of the 90s, and not the language you should be writing your deliverable in. Unfortunately, python happens to work well for getting something functioning, and turning that into something that functions well gets called "maintenance". I've given up that fight and now I'll just stay in python land for as long as it takes to isolate it into a separate process and then go and write the rest of my code in something I prefer.

It's not the only language I've used, though, and I've had basically the inverse of your problem from time to time: people who think running an API-documentation generator over their statically-typed code, so that it just spits out a big list of functions with their return types, parameter names and parameter types, is "documentation".

Now, it may be you come from a world where that's the standard way to do things and you've gotten used to it and mostly don't notice just how much further research and digging it takes to figure out from "Frobnitzable bunglify(Widget[] w, int c)" what it is exactly that it, um, does. But it's hard to knock people for not writing documentation when advocating for a language feature that encourages people not to write documentation!

My experience is that it's easy to fail to produce useful documentation in just about any programming language, and that languages' choice of type systems does not correlate significantly with the quality of documentation of key libraries and tools.

What tool are you using in which completions take so unpleasantly long?

[0] - https://github.com/gorgonia/gorgonia

[1] - https://github.com/chewxy/hm

Swift overall seems like a good possible static language for ML, but gaining popularity is a slow thing. Actually I really can't seems to see much other alternative languages than it that could gain popularity except maybe Julia.

Then you’re in luck, because that language is R!

The problem might be that many people doing ML don't really know how to program beyond a basic level, and at first glance a language like Python seems "easier". But they get by anyway since the difficult part of ML is often not really the programming as such, and if your program is just a 100 line script calling Tensorflow then any language can work, however annoying it may be.

Absolutely ... I feel like the ML world is caught in a strong local minima with Python: it gets people on board due to the exploratory ease, but then the effort of learning a different system once they already understand things in that ecosystem is just too great. Unfortunately it is highly suboptimal for writing highly structured, complex code.

You should have a look at DL4J and ND4J if these things frustrate you. Particularly with a dynamic JVM language like Groovy they give you something close to the best of both worlds.

I don't buy this argument. If you want your program to do what you intend you have to know the types anyway. I've never spent significant time on finding the right type, even in Rust.

I have spent significant time on incorrect code, because of functions that accepted inputs with the wrong type and didn't complain, because the wrong type had the same method.

I have spent significant time on looking up what a function can do, because the IDE can't figure out what my parameter is.

I have spent significant time on programs that ran into runtime problems every once in a blue moon that would have been easily detected by a good type system at compile time.

Articles like this just show me that every language that needs to have maintainability for large projects will turn into strongly, statically typed at some point. Whether it's type hints or TypeScript.

This is not the indisputable fact that you seem to be portraying this as. I've written similar amounts of Rust and Python, and it is much much easier and faster for me to write robust software in Rust than in Python. Python might let me get to a prototype more quickly in some cases, but the amount of time spent on maintenance after the fact puts Rust very clearly in the lead in my experience.

There's no doubt that Rust's learning curve is much much steeper than Python's, but this is a very different thing that saying something general about overall development cost.

I like both, but Rust and Swift are really for two very different groups and solve different problems. Rust is for the C/C++/assembly programmers who would never even think of using Java or Python. Swift is a modern Java.

If you're coming from Python, I think Swift is closer to what you want since Rust will probably seem more complex for reasons that don't have benefit for the Python-style use case.

The only downside is you have to learn about a (fairly straightforward) type system. But welcome to almost every other language.

A typesystem is a limitation you place on your code which you accept in the hope that the things which it proves true (problems it avoids) are worth the limitations that you've accepted. You can set everything to Object but the culture that develops around statically typed code tends to mean that people are really afraid to ever do this or be ostracised.

From a data moving point of view there's a couple of examples I hit in production where a typesystem isn't helping me => * DataFrames tend to gain/lose columns on every operation, so to represent them in types I'd need a zillion types with code duplication or an interface for each field name. Some operations aren't even possible to type like pivot_table. So tons of my code is really not helped by typing * Everything turns into a DAG of data flows, you can do this with typing but the overhead would mean that noone woudl come up with something like mdf (I don't use mdf in prod but it's impressive for specific applications)

If you want a better defense of dynamism google Rich Hickey, a point he made:

There's many choices to be made for typesystems, you can have generics/not, higher kinded types, dependent typing. Maybe it's better not to enforce these choices right at the root language level and allow them to happen at a higher level (be plugged in).

https://stackoverflow.blog/2017/09/06/incredible-growth-pyth...

Most programming I've done in the past has been with python (no type hinting) or languages with weak static type systems (java et al). Having to verbosely specify the types in java was always just painful, while the upfront speed and freedom of python felt much nicer. However after getting a taste of better type systems by dabbling in rust and elm, now every time I write something in python I'm thinking "was argument a string or a foo class?" or "that runtime error would not have happened if the type checker caught it".

Python type hints now are a huge plus for the language. It still seems a bit "hacky" though. I'd love to see something that adds more syntax sugar and feels more built in. Maybe it's just me.

I find this sort of generalizations off-putting. It is all about trade offs.

Types can be very useful. There are whole categories of errors that are avoided by types.

And yet, in many cases they are a productivity drain. Time is often better spend by writing testcases instead of chasing down type issues(more so if a language has a 'generics' or C++ templates concept), so that approach got favored in many fields. These will catch type errors, and static languages also need tests, so the gains are not so easily quantifiable.

It is not just a matter of ergonomics, it is a matter of getting out of the way of the problem domain. Most people are paid to solve actual issues, not to craft convoluted C++ templates, custom types or deep type chains.

Type inference helps, but even Haskellers will often avoid it when defining functions, so that they will be explicit. But Haskell has incredibly powerful constructs. I was amazed that you can actually tell the compiler you are writing a recursive data structure[1], which is something that most languages won't allow you to do – the most they will allow you is to define a reference to the same type of object(or 'struct'), and you have to stitch that together in runtime. Simple cases are also simple. A function that gets a list and returns an element of that list? [a] -> a, I don't care what a is, nor does the compiler. But feed it something that is not a list, and it will complain.

A language should let you easily define new types, and allow these to be used without friction by standard functions. For instance: if I have an integer, is this integer describing a temperature value? If so, functions that work with speed should refuse to operate with this data. Standard mathematical functions _should not care_ and should do whatever operations we request(without any silly type casting, I told the compiler this thing is a number, use it). The compiler optimizer should also use this information to generate appropriate assembly instructions.

So yes, if you have an expressive type system, types can be a joy (you'll find yourself thanking the compiler often). If instead your type is spent trying to figure out why IEnumberable<Something> cannot be converted to IEnumerable<SomethingElse>, even though they are subclasses and even though the same operations work with arrays, then types are getting into the way of solving real problems[2][3].

And even with an expressive type system, there is a non trivial cognitive cost. Sometimes you just want to write (map (lambda (x) (do-something x)) lst) and not really care what it is that you are manipulating. On the other hand, I personally do prefer using Golang to talk to AWS apis (instead of python or javascript), precisely because of their deeply nested type hierarchies.

Being able to sprinkle type annotations where it is useful and have the compiler figure out as much as possible otherwise is a good compromise, which is the approach that the Python article talks about.

I would read this as not favoring mandatory static type checking in 2018 but, instead, using a hybrid mechanism that benefits from both approaches.

[1] http://tuttlem.github.io/2013/01/04/recursive-data-structure... [2] https://blogs.msdn.microsoft.com/rmbyers/2005/02/17/generic-... [3] http://www.baeldung.com/java-type-erasure

Type inference might got one halfway there, but it would work best at the call site -- the type of variables could be inferred from return values and from literals and so on. But you'd still want to define a function as basically returning "Any". This means a caller wouldn't have a type for the return value, and so if you want to do "fileName := funcReturningAny(); openFile(fileName)" and openFile() was defined only for a string argument, the compiler would have to generate a runtime check at the call site to assert the type fileName as being a strong. This the call site is penalized for being typeless, but the statically typed code wouldn't be.

I suspect this would work pretty well. You can start out with no types, and you'd be able to add types as you go, slowly tightening the screws on your program. Generics, traits and other modern abstractions would still be available, though some of those features would start to become unavailable or awkward to use with any code not sufficiently tightened with types.

I don't know of any language that does anything like this, although I suppose TypeScript is pretty close.

If anything adding a list of assertions at the top of a method/function/class is much more readable than the current type hint implementation.

And if you're running out of resources move the assertions to your regression tests.

Do you have a specific project in mind? I know of a few, like Cython, RPython, Nuitka etc. that already do this, but use their own type annotations that are partially incompatible with the Mypy type system.

One hurdle to native code generation is that most large Python projects will contain a bit of metaprogramming that the type checker can't handle. That isn't a problem if you can check it by hand and use assertions to signal the type checker that you know what you're doing, but that code will also be impossible to compile. There are some ways to get around that (e.g. RPython allows arbitrary Python during initialization, which is executed at compile time), but they might require a complete rearchitecturing of the program.

    a: int = 1