What’s New in Python 3.5
docs.python.org
docs.python.org
The problem Python faces is that Python's Little Tin God painted the language into a corner. He insisted for years that everything in Python had to be fully dynamic. This limited performance.
Then came competition in Python's space. Go came along, Javascript got traction on the server, and Python faced a real threat. Both of those can beat Python on performance, sometimes by huge margins, and they're useable by Python-type programmers used to the freedoms of scripting languages.
The Python 3 debacle had already thrown the Python community into disarray - most of the production work is still on Python 2, which was supposed to be abandoned but refused to die because the cost of conversion was much higher than expected. Meanwhile, Python 3 takeup was far less than expected; numbers like 10-20% come from downloads, and production use is probably less. Another incompatible change in the syntax would be rejected by the market. An incompatible change to a "typed Python" wouldn't fly.
So we get PEP 484, which is ugly but might help Python survive. Might.
In any case, JavaScript is dynamically/unityped, so its performance is unlikely to provide a reason for type annotations. The real problem here is that PyPy is still maturing, and can't be used by many people because of legacy C extensions.
This seems really hyperbolic. I'm not sure Go really competes in the same space as Python; also Python is so well entrenched as the successor to Fortran in the scientific computing space, that this would seem to guarantee continued relevance far into the future. It's probably like a lot of things, it may not be as sexy or break neck fast (though PyPy is staged to change that, and maybe we can also feasible get a non-sucky version of IronPython or Jython), but it's pretty much everywhere now, kind of like Perl, C, PHP, Java....
Uhhh... no. FORTRAN is literally the fastest programming language in existence. Maybe Python is good for prototyping or pre/post-processing some data, but with a performance hit in the 100x order of magnitude, you won't run Python scripts for the bulk of any serious scientific computing project.
[1] http://docs.scipy.org/doc/numpy-dev/f2py/getting-started.htm... [2] http://pymca.sourceforge.net/
Also: http://www.quora.com/How-did-Python-dominate-scientific-comp... http://programmers.stackexchange.com/questions/138643/why-is...
Finally as a Physics person, you might also like the coursework for this (I certainly enjoyed the course): http://pages.physics.cornell.edu/~myers/teaching/Computation...
I don't need raw speed. I need development speed so I can easily iron out bugs and try new methods. My colleagues develop in R for the same reason.
Stopped reading after this.
My housemate is a Math/Stats person he works in finance and uses R for much the same reasons.
Maybe Python is useful but it would have to offer me something compelling to make me switch over.
Your criticism of Python being dynamically typed is also misguided. There are many benefits of having a dynamically typed language, which I won't bother to enumerate here since this subject gets beaten to death regularly on HN. It is a good choice to make this design decision up front and honor it as the language grows. Guido is not a moron; he knew there would be performance implications. Nobody today chooses Python for it's native performance anyways (although Cython and PyPi have made great strides for common cases).
The value of Python is not in performance, it's in the language simplicity, large ecosystem, and highly developed libraries. There are some disciplines such as machine learning and quantitative finance which are all but predicated upon Python, with excellent results. Comparisons to Go and JS are incongruous; those languages have other benefits which would make them good choices if things like concurrency (Go) and very high level abstractions (JS) are important.
The Python 3 transition was indeed rough, but in no way is it a "debacle". The community is not in "disarray"; that's absurd. The transition to 3 will happen eventually, and indeed this lethargy was caused by deliberate breakage in language features. Maybe not the best decision in hindsight, but far from this cataclysmic fantasy you seem to be depicting.
It's astonishing to see how fast Python is moving in this direction. It is truly a powerful language for async computations now - surpassing in this ability both JavaScript and C# (at least for now). For example Python got `async with` but JS isn't even close (userland solutions like Bluebird's using exist) and C# is only starting to work on `IAsyncDisposable`
I want to ask:
- Does it mean that it will be possible to create realtime apps with python? Will it be possible to use it instead of node.js? Do you think Django will implement that functionality?
- Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python?
That would be so cool....
I'm not sure what you mean by realtime but usually real time means something else and requires more deterministic behavior.
If you mean applications that use non-blocking IO to great extent then yes, but this depends a lot on the ecosystem to provide libraries for async file io, database io and so on. The Python language left good infrastructure to use though.
> Now that WebAssembly is coming - doesn't it mean that it could be possible to create both backend and frontend for realtime apps in python?
Not anymore than we could before with asm.js unfortunately as seen in http://repl.it/languages/python3 the problem is wrapping and the library, I hope a company "picks up the glove" and makes "Python for the frontend", sharing Python code between backend and frontend could be amazing.
It's been possible to use Python instead of node.js since before node.js existed. And by "possible" I mean "tons of people have been shipping code", not just "it's theoretically possible but nobody does it".
Node ported a well-established existing technique to Javascript, it didn't invent it.
I'm thinking of how in Go I can launch a goroutine with "go foo()", where foo can be any function and not one that happened to be declared explicitly as async.
Or in Perl 6, I can "my $p100 = start { (1..Inf).grep(*.is-prime)[99] }; say await $p100;" This provides the similar blocking "await" keyword, but I'm still free to execute any arbitrary code inside the block denoted by "start".
I have a feeling that these questions are both answered by whatever coroutine functionality existed prior to the introduction of "async def", but that's an area of Python that I have no experience with. I'd appreciate any clarification that could be offered. Thanks!
Edit: re: the P6 example, it looks like it works somewhat like Python, where a "Future-like object" (from the PEP) is provided to await; in P6 start returns a Promise. However it looks like in Python I still need to wrap the enclosing function with "await def". https://www.python.org/dev/peps/pep-0492/#await-expression
Yes. One main one is splintering the library ecosystem. So some libraries are using Twisted, some are using base threading system, some run in Tornado, some will use async and so on. They are usually not mixable. One day in the distant future they will all support async, but that means updating all of them.
Special markers ("async" waits points/objects) for IO co-routines tend to propagate vertically through all the API layers. If, say, a low level library you found is returning a Deferred/Promise/Future or generates values, then top level has to handle that as well. Then its parent also has to handle it.
It is infecting the ecosystem with low levels details about how the IO needs special casing because of how the GIL works. Yes, it is more elegant in Go, Rust, Erlang, C#, Java etc.
BTW Python has something like it based on the greenlet library (eventlet and gevent libraries are based on). Those monkeypatch system libraries so it manages to hide this complexity, but there are other costs, unsupported or untested side-effects being one.
See this:
http://dabeaz.blogspot.com/2010/02/revisiting-thread-priorit...
Or even better for a demo watch David Beazley's Pycon 2015 video (It is an awesome video even if you don't care about the GIL or socket programming).
http_request(url, callback)
def callback(result):
url2 = get_from_value(result)
def inner_callback(result):
print('Final result:', result)
http_request(url2, print)
http_request(url1, callback)
With async/await you could do something like:
result1 = await http_request(url1)
result2 = await http_request(get_from_value(result1))
print('Final result:', result2)
This model is more explicit than the Goroutines of Go or the multiple threads of Perl 6. But it has the advantage that the user has more control over when the execution control flows between different parts of the program. For example, if I have this code:
await request1()
# some synchronous code
await request2()
async def http_request(url):
# ...
await http_request(u)
In Python, the closest equivalent to `go foo()` is `threading.Thread(target=foo).start()`. Or you can use gevent to get goroutine-like lightweight threads, but I find this is works better in go than in python because the entire language is designed around it. In python it's common to find libraries that are not compatible with gevent's monkey-patching and so you lose concurrency in a way that is difficult to detect or guard against.
If code is using the regular Python socket and threading libraries, it will almost always work seamlessly. Gevent provides the closest thing to goroutines.
I suppose you could follow the .NET Naming convention of appending "Async" to all routines.
def moo():
return 5
async def oink():
return 5
For a human, without the 'async' marker, you'd have to scan the entire function source to know. I guess also type checkers and documentation tools like to have an idea about the return type.
def important():
await something()
def something():
# ... stuff ...
await something_else()
# ... more stuff ...
[1]: https://www.python.org/dev/peps/pep-0492/#importance-of-async-keywordWhile the format() mini-language is nicer in a lot of ways, the convenience of % and its ubiquity mean that I generally use it more often. The main time when I reach for format() is when I need more detailed control of column widths and the like, which you can do with things like "{var:{width}}".format(var=var, width=width)
Edit: I think it's largely the special flower overloading of the % operator that bugs me the most. If % was spelled .sprintf I'd like it more.
I still wish they were consistent and provided a .format method for bytes, especially since, as you point out, .format was supposed to be the "new way" to do formatting and this feels like a step backwards. And what about backward compatibility for porting Python 2 code that used .format to do formatting on what are semantically byte strings?
[1] https://www.reddit.com/r/Python/comments/3c7lne/python_350b3...
[2] http://thread.gmane.org/gmane.comp.python.devel/144712/focus...
Python 3.5 may actually be what finally convinces me to move to Python 3 (that, and the end of Python 2.7 support coming up in a few years).
The new async stuff, finally fixing some of the problems with byte strings that made them not really an adequate replacement for Python 2.x strings, and os.scandir is a substantial performance improvement over os.listdir plus calls to stat (though os.scandir, at least, is also available in Python 2.7 via PyPI).
True. It might even outlive Python 3, just as Perl 5 looks like it will outlive Perl 6.
Unless there is some major breakthrough in porting tools (in combination with more and more libraries supporting 3), Python 2 will still probably be the only Python version in many enterprises for the next 10 years.
Everything dies.
> Ask Kenneth Reitz for example. He says he will use Python 2 forever.
So, assuming this currently-stated intent never changes and Reitz is immortal, Python 2 will always be in use.
The first of those premises is somewhat suspect, the second even moreso.
> There are a lot of companies with huge codebases who never will port to Python 3.
That's probably true. Its less likely true that those companies will never port off Python 2.
Python itself might die when Python 2 dies, and arguably, Python 3 makes that more likely.
For example, in one performance intensive code that I use, while mostly written in Python, has a C extension, which itself uses inline assembly for the core routine. Similarly, most numeric calculations that use NumPy will end up doing BLAS / LAPACK / ATLAS calls, so the performance limitations aren't due to C. The switch from 2 to 3 isn't really going to affect these.
If it was seriously and consistently worse, eg, for Django, then it's likely we would heard about it by now.
Instead, there are reports like https://www.youtube.com/watch?v=f_6vDi7ywuA (slides at https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-... ) from PyCon 2013. At minute 30:54 / slide 16 the speaker evaluates the two and finds that the median performance was effectively identical for microbenchmarks, and ~5% slower on macrobenchmarks, though even then it varies from 27% slower to 13% faster depending on the specifics, and as the speaker points out, the tested packages aren't doing exactly the same things on both versions.
In any case, that was 2 years ago.
Alternatively it's insanely easy to spin up a Linux VM with vagrant and just compile and run python scripts from there. Everything will pip install without any issues.
Or between the new async / await and the current asyncio?
It looks a lot like the one on py3readiness.org now, and it doesn't have all the nonsense like "tiddlywebplugins" that it used to have.
I remember Armin Ronacher (author of many nice things, especially Flask) even wrote a blog post against Python 3 and this was one of things he hated about it.
> APL apparently used +.× , which by combining a multi-character token, confusing attribute-access-like . syntax, and a unicode character, ranks somewhere below U+2603 SNOWMAN on our candidate list.
and then later, one of the reasons @ is better than the alternatives:
> Whatever, we have to pick something.
(Aside: HN really needs a better way to escape star inline in text.)
Given that having an operator for this was deemed important for more readable numeric code, and that * was already taken, there weren't too many fabulous choices left.