Python 3.5 type hinting in PyCharm 5
blog.jetbrains.com
blog.jetbrains.com
def greeting(name: str) -> str:
Turns out to be backwards compatible in Python3 before 3.5 (though not Python2, yet another reason, if you actually needed one, to start new stuff in Python3)This is one reason. All my personal fun-type projects are in the latest Python, and I have the leeway to update work to this.
I know there are a lot of people who don't like or want this, but coming from a C# background before Python, I really like this feature. (not that I want Python to be more like C#, I just like the safety net and guaranteed documentation.)
EDIT: A gem i found on Reddit: Configure setuptools to install it only if required².
They are just arbitrary expressions (though in Python, type names like 'int' or 'str' are themselves objects). They don't do anything by default but are accessible as fields of the function object.
A lot of third-party libraries cropped up that used them to type-check stuff. Type-checking was a popular enough usage for them it's official now, to help gather people under the same libraries and terminology.
You can still use the annotations for other things, of course – the dictionary is still attached to the function and can be inspected as before – but if nothing else is specified, we can now assume they are type declarations.
Addressing pain points is unfamiliar to the Python culture?
In practice, type intransparency is a serious problem – duck typing is only fun as long as everyone a) knows what a duck is and b) agrees on what it should do. Type hints make it a lot easier to work with opaque libraries and inside large codebases.
On the contrary. I use Python professionally since 1997, and it has had all kinds of features added on for different needs, from decorators and generators to async and virtual environments, context managers, list comprehensions, dict comprehensions and what have you.
Of the languages I know of, only C# has had a similar pace of adding new stuff in to the core language (not just libraries).
When did Python shy away from adding new features?
It seems that Python didn't try to please only 2 categories of people: those who dislike significant whitespace, and those who want proper closures.
>and kept a coherent design philosophy that resulted in a clean and simple language
At this point I'd say that Python, with its full feature set, is as complex as a scripting language gets. Maybe Perl 6 trumps it, not sure -- but what you describe is closer to a language like Lua, or tcl or even Go than Python.
Uh, ever seen "Batteries included" referenced in their documentation?
def (str) greeting((str) name):
Edit: it was just an idea, it would be a bit more C-style syntax; of course backwards compatibility wouldn't be possible. Instead of downvoting would you care to explain?1. Instead of being more like C-style function declarations, they look a lot more like C-style type conversions.
2. However closer it may be to C (type before identifier I guess?) it's pretty far from any other mainstream language. For instance, C++ has support for -> syntax for function types (http://stackoverflow.com/questions/22514855/arrow-operator-i...) and some other languages (I can think of Scala) use name: Type declarations.
As a personal preference, I find the python type hinting more familiar than your example, despite (myself) being most familiar with C-like syntax.
Putting the type after the name feels more natural, and the : also translated nicely into 'of type' (or if you prefer for functions 'returns type').
For familiarity, while the C family of languages favors "type name" declarations, the Pascal and ML family of languages use "name: type" instead. So, familiarity is entirely a function of what language you used before.
Second, C-style declarations have their problems, which is why they aren't used much outside of languages that derive from C. Generally, you look up declarations by searching for the name of the variable to determine the type; this generally has less cognitive overhead if the variable is at the start of the clause, not the end, ideally separated by punctuation (note how the "name: type" syntax mirrors Python dictionary "key: value" syntax) [1].
[1] Also, in C/C++ the "type name" syntax complicates the parser, as the parser has to be able to distinguish between type names and other identifiers; while this wouldn't be an issue for Python, there is still no reason to perpetuate this syntax choice.
This was talked about and was there for a while.
https://www.python.org/dev/peps/pep-3107/ [2006!]
These weren't specifically called "type" annotations but I think that was in the back of everyone's mind.
greetings.__annotations__ = {'name': str, 'return': str}
Same exact effect.The syntax isn't the magic sauce, the type checker is.
def greeting(name: str): str:
or: def greeting(name: str): str
Using once ":" and then "->" for type declaration is inconsistent. Not to mention that the dash is not always aligned with the tip of the ">" and it looks ugly, not like a real arrow!The IntelliJ IDE set is really far better than I understood. IMHO.
#greeting :: (str, int) -> str
def greeting(name, age):
In fact, I wonder why more languages don't adopt this syntax, since it greatly simplifies functions with very large types.Anyway the haskell way, especially with Typeclasses, has to be clearest way to express type constraints for functions I've encountered at least from a code readability standpoint.
#greeting :: str -> int -> str
and this is specifically related to the functional nature of Haskell. `greeting "hello"` now instantly becomes a function with type :: int -> str. So the Haskell syntax is about currying, it's about how the function you wrote is really just a cascade of functions.
Doing something silly like making the "arguments" reside in an artificial tuple would definitely be bad practice in most Haskell cases, and so syntactically looks a bit weird and out of place, like it's shoehorned on just because Python function arguments are not like Haskell arguments.
Since Python is not based on this model of a function, I think it's a lot better to keep the syntax in the actual def statement. Adding the return type annotation at the end is nice.
Most probably, this stuff was added to the signature because there were already natural tools like the inspect library that would provide the immediate hooks for making use of it. But even if the implementation didn't need it to be part of the signature like that, I still think it's better for Python's paradigm.
However there are also cases in Python where you get currying, like decorators. And for complicated types like:
# greeting :: (str -> (str, str), int -> List[str]) -> Dict[str, int -> int]
being able to separate the signature from the declaration certainly makes reading it easier.Just for example, in Python it's extremely common to use default arguments, and in fact a lot of stuff in the typing module exists to make it easier to do something like make an Optional type that can either be `None` or a `str` or something. So whatever type signature you're going to mimic from Haskell, you won't be able to give optional types or default arguments without making it totally un-Haskell. And of course no one is going to go off and change Python to work based on Haskell-like type classes (nor should they).
http://mypy.readthedocs.org/en/latest/type_inference_and_ann...
And specifying the concrete str type goes diametrically against the "don't check against a concrete type, embrace duck typing" mantra that Python programmers are taught early.
Because you would need to extend the base list type to support indexing just for this feature, which is a bit iffy if you ask me. I think you have a point about "don't check against concrete types" though, they should update the docs to suggest using an 'Iterable[type]' over a 'List[type]' unless you really really really want a list.
(author)
the type hinting in Python is just that -- hinting, right?
It's for IDEs like PyCharm to use to generate better docs and to be more intelligent about suggestions when writing code. But what it does NOT do is inform the interpreter/compiler about type information, correct?
I.e., Python3 code with perfectly implemented type hints would only make it easier on the developer, and not the interpreter/compiler, yeah?
[1] https://github.com/williame/obiwan
What I do is I have my test runner look for the PYTHON_TYPECHECK environment variable, and if it's set, it imports and enables Obiwan.
I've caught a few subtle bugs this way.
Are there a any good libraries to turn on/off type checking using if debug as part of the decoration logic? If so then running with -O in production would solve some of the problem with this.
http://doc.pypy.org/en/latest/faq.html#would-type-annotation...
At the moment no. But there's nothing to say that it couldn't happen in 3.8 or 3.11 or 4.x, since the groundwork is there now.
def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None:
people are going to get them wrong some of the time. Dumping the checking problem on some IDE, with a very vague specification of what an error is, is not progress. If functions have type specifications, you should not be able to call them with the wrong type, ever. This is the job of the compiler and the run time system, not the IDE.Somebody should stop Python's little tin god before he creates another giant step backwards like the Python 2 to 3 mess.
There aren't.
> whether the standard build tool for the language will run that tool on a proposed library release before that library gets uploaded to PyPI
It's optional. If the library maintainer wants to do that then he can.
A library with no annotations in PyPI is fine-ish. A library with correct annotations in PyPI is fine. A library with incorrect annotations in PyPI is extremely not-fine.
I'm fine-ish with annotations being optional, but if they're there then their correctness needs to be enforced, otherwise they're worse than useless.
As is a library with broken logic or one that throws exceptions randomly. You want Guido and the developers to implement some kind of system that automatically scans all PyPi packages for any kind of errors? I'm sure them solving the halting problem won't take that long.
Incorrect annotations are a bug. The onus is on the maintainers of packages to fix bugs in their packages, not PyPi which is a distribution channel.
Yes I do, but perfect is the enemy of good; it's ok to do something that makes things better without solving the whole problem. I want the build system to encourage people to do the right thing. E.g. any serious build system already ensures that you run the unit tests as part of the build or at least release process, because it would be just dumb to publish a library release on PyPI if it failed the unit tests.
> I'm sure them solving the halting problem won't take that long.
We already know how to avoid the halting problem, it's called types; see e.g. Idris.
But you don't need to solve the halting problem to check annotations. It's not even hard. Why would you want to not check them? I mean if you're not even going to check them then why bother having annotations in the first place? Comments would provide the same functionality and have less danger of being misleading, because everyone already knows comments often end up being out of date.
All you seem to be saying is that "Code should be tested before being uploaded to PyPi", and somehow that equates to type annotations being bad? Again, it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu), and yes this should include a linter to ensure type annotations are correct if they are using them.
I'm saying that the build tool should run unit tests and check annotations before uploading a package to PyPI. It doesn't matter whether checking annotations is done by a "compiler" or some other tool, but it does matter that whatever tool that is runs as part of the language standard release process (and in practice it tends to be the case that when a compiler enforces types then this happens, and when an external tool does then it doesn't).
> it's up to the developer to ensure his tests pass (by using a ci, or running tests locally, or even making a blood sacrifice to Cthulhu)
Leaving it up to the developer is a recipe for it not happening. We're programmers, we should automate these things.
Building in Python is little more than creating a .tar.gz (or other format) of your source directory. Testing is very different and is very specific per project.
Anyway, this whole discussion is pointless, you just need to run 'python setup.py test' before you run 'python setup.py bdist'. Is that so hard? If you really want you can edit your setup.py to do this first.
I personally take a sort of halfway approach: I use Travis heavily for public documentation of the fact that my packages are fully tested and passing, but stop short of having it automatically release to PyPI for me when I tag a new version.
But in general, the tools can't fix the fact that PyPI is not heavily curated. Any non-curated package repository will have the issues you're complaining about, even if the tools default to doing what you want them to do.
For example, C would have been a much safer language if lint was part of the compiler, instead of being optional.
"This is the job of the compiler and the run time system, not the IDE."
Making this optional will mean the majority won't bother to run the checker.
How many do bother to run lint, jslint, core.typed or dialyzer?
But those are usually the same ones that take the effort to run the tools anyway.
Having most of the standard library plus large portions of the ecosystem come with these hints will be hugely beneficial for productivity.
And of course, there IS a way to check them. IDEs, linters, etc. - I'm sure if you say that your function returns an instance of User and then you try to return a boolean, they will yell at you. And I expect that the standard library and popular libraries WILL run these tools before releasing a stable version. A 90% percent solution is still a step forward compared to relying on much less reliable human brains at every step.
The entire point of type systems is to enforce correctness. They allow you to describe problems on the type level and ensure that the code being described matches that description, eliminating certain classes of bugs. That's why languages have them, that's the expectation.
Type hints are just an overhyped standardized comment format with special syntax support.
I still think that machine-readable documentation can be immensely useful when combined with a static code analysis tool. Not as useful as "real" static types would be, but useful nonetheless.
It is the exact same idea as Microsoft's SAL annotations for C/C++ - https://msdn.microsoft.com/en-us/library/ms182032.aspx, or Java's @NonNull, @ReadOnly, etc. annotations.
This assumes you can represent a type specification for the function you are calling, this isn't always possible in a dynamic language with advanced metaprogramming facilities.
Think of it as modularizing the compiler and runtime so that the dials on dynamism can be tuned by the team.
PyCharm already used pretty sophisticated type inference this just helps it. But the real win is being able to run mypy or pytype over a codebase automatically.
Types, Contracts, Properties, Tests are different directions in a tensor field. With Python you can mix and match at your will. This is in _no way_ relatable to the 2-3 mess.
I can have real static typing if I want it. I can also have dynamic typing if I want that. I can use lambda functions. I have LINQ. I can use ScriptCS or some of the new Roslyn stuff if I want to use it as a scripting language. Unix support is not as great, but they are coming along with that. There's no GIL.
Is there some killer feature for Python that I'm overlooking?
Still on average more concise to express the same idea.
Standard library is a nice mix of function-based vs. class-based solutions where appropriate, rather than forcing OO on you even when it's not the ideal approach.
Cross-platform C# usage is still not even a reality, let alone as good as what Python already has.
In several domains of programming, Python has best-of-breed, and sometimes just flat-out the best, available libraries.
Now that Microsoft is supporting non-Windows with .Net core, C# is more attractive. That said, if I was thinking compiled Python with static typing my mind would go to Go.
Lots of options!
Python use cases are completely different from C# - Python is all about fast iteration and prototyping, REPL, interfacing to native libraries etc.
C# is about static typing and all that it implies.
F# is slightly closer to Python with REPL and type inference but still on the C# side of spectrum.
I sometimes wish that one programming language would work for all of my requirements but I keep needing: scripting languages for quickly getting stuff done; practical languages like Clojure and Java, and type strict languages like Haskell (that I would like to use more, but development takes longer).
Looking ahead a few years, the ideal for me would be a loose scripting language that had awesome static analysis tools. So, for example, I would get the safety of Haskell when hacking in Ruby.
For those of you using vim, emacs, and the standard UNIX editing toolchain, the mypy linter provides command-line support. This blog post from April 2015 describes the first version of mypy that includes support for the official typing library.
http://mypy-lang.blogspot.co.uk/2015/04/mypy-02-released.htm...
Having recently switched, PyCharm really does make larger projects a lot easier to refactor / manage.
Edit: Also, to actually answer your question, this is a relatively new feature so I don't think any of the VIM plugins support it yet. Jedi has had type hinting from doc strings for some time now. I'd imagine they'll update that to use this soon.
http://stackoverflow.com/questions/25588642/pycharm-false-sy...