Reasons for Python async/await
lwn.net
lwn.net
It's a really exciting proposal. AFAIK, it'd make Python 3 the first mainstream imperative scripting language to have these async/await concepts built-in (there is a similar proposal for JavaScript/ES7 built on top of generators and promises[1]). While I've commented before about how asyncio is still a bit more nascent than I'd like, I'm really excited to see where the community takes it.
("More mainstream" is a moving goalpost fallacy.)
Of course the "mainstreamity" of languages is hard to determine, but Dart is so low in any kind of related metric that makes this a non-issue.
Let's put it this way: the Dart team announced a few months ago that they'd be concentrating on transpiling [1] and they won't put the VM officially in Chrome, and nobody gave much of a fuck judging from reactions in blogs, forums, HN etc.
If a mainstream language changed something, even something minor, there would have been blood (as it often happens).
[1] http://www.infoworld.com/article/2902074/javascript/google-d...
I don't at all mind your saying so, but it might be frowned on around here.
> Mainstream is not a "moving goalpost fallacy".
Why can't you read?
If I write songs that sound like Taylor Swift but no one knows who I am, do I make "mainstream" music? Personally, I would say yes.
You might make mainstream music but you yourself are not mainstream.
http://www.tiobe.com/index.php/content/paperinfo/tpci/index....
Surprised?
Dart is hardly used by anyone.
Currently there’s a large split between folks programming against Twisted, Tornado, Eventlet/Gevent, rolling their own thing, or just writing blocking code and getting crap performance. (Not to mention Stackless, etc.)
These will all continue to work going forward, but they will now be interoperable and based on a common foundation which is solidly implemented and included in the standard library. Everyone wins.
The one feature that IMHO should get baked into a language but nobody's done right yet is API versioning. Wish more people paid attention to the evolution of a program over time rather than a snapshot of where it is now.
I don't think this is strictly true. If there is code which is available only in async form, then it's either unusable or awkward to use at best for someone using the non-async form. And vice-versa.
This is something I've found to be a problem in C#. Specifically, trying to use Microsoft async only API from sync code. And that async API really requires the whole stack to be async, so using the APIs from sync code requires hacks found at Stackoverflow.
It's a split in the code base, and extra work and complication for developers. I'm not saying that it's not worth it, or it shouldn't go in.
This is already the case today. As I pointed out, there is currently a split into about 4–5 (at least) mutually incompatible versions of I/O and network protocol related code. The new features should cut that down to 2–3, which are easier to understand and reason about, to boot.
When I say people can ignore the changes, what I really mean is: to the extent they can ignore the complexities of syntax and semantics around asynchronous code today, they can continue to do so in the future.
Alternately, having language support and a solid built-in library might encourage some people to write their code using asynchronous rather than blocking I/O, without too terribly difficult a learning curve or too much mental overhead from the new abstractions.
While I love Python2, my Python3 will be Go.
I like that Go is not a kitchen-sink language. I've also learned more from it, and I believe it adds more to my skillset and resume than Python3 ever will. That's my plan at least, to stop using Go just for fun, stop using Python for daily work, and use Go for fun and daily work.
That's when the time comes that Python2 is no longer relevant and doesn't have the library support it enjoys today. I expect 2.7 library support to last a very long time.
Having said that, porting your stuff to Python 3 isn't that bad. The automatic conversion tools get you 99.5% of the way, unless you're doing really crazy stuff.
I work in one of the largest Python shops in the United States. We don't have 100% test coverage (I don't know anyone who does). So there will be stuff breaking. With hundreds of thousands of lines of Python code, the testing/fixing burden would be enormous. It just doesn't make business sense to move. This codebase will never be on Python3. And I think it would be a serious mistake even if it could be done, as the Python3 "migration" isn't done, until it's actually happened. Right now the numbers I've seen are that 10-20% of the Python userbase is on 3.x. The future for Python3 is sketchy at best, those support dates are nothing more than political propaganda. If they meant something, they wouldn't have already pushed it back to 2020. Even if it did matter, code doesn't just stop working when the ball drops in 2020.
So if I could snap my fingers and port us over to 3, and have no breakage (which would cost us millions if not sink the entire business)- I still wouldn't do it. The best strategy is keep on using Python2, which with ~80% of the userbase still at this point in 2015. Python 3.0 was released late 2008. It would be a mistake to switch even if it were risk-free.
We will be using Python2, and bringing Go in-house for new components where we would have used Python3 in an alternate universe where the migration wasn't botched by the CPython core dev team.
Go mostly adds a lot of busywork for exchange of a speed bump (that you could get with another decent language with types, like D or Nim, if only they weren't unpopular).
Nothing about Go makes someone a better programmer the way learning C or Lisp or Haskell does.
Should be plenty of time for that to happen. There's probably a few decades of 2.7 library support, given the stats I've seen show 3.x at 10-17% of the userbase. Even if that doubles in 5 years (won't happen), it'll still be ~34% on Python3. Which is so long from the 2008 Python3 launch date, and a small enough number migrating that we can effectively concede these are two separate communities going forward.
So what should an end user do? Build on 3, which still has many things missing and less reliable 3.x libraries? Or stick with 2 which has everything the 3-only ecosystem has and much more. 3.x in reality makes no sense unless you have a political bone to pick. The big attraction was the unicode by default.
Python has supported unicode since 2.6. Python3 presents a new default way of handling it, nothing more. Py3 is pure technical churn at its worst. There's no innovation there. As a result the users outside of a vocal minority aren't coming.
While I think that was the right decision at the time, for me the balance shifted about 6 months. Needing access to the much improved multiprocessing library in Python 3 I ported my application to polyglot with the help of the python-future library. It really wasn't very difficult.
This was a modern codebase of about 20K LOC with reasonably high test coverage. For much larger or older applications then not porting is probably the right choice. But for new applications I think Python 3 is a fairly good bet now. Even if only 15-20% of the Python ecosystem have switched then that's still a significant enough userbase for most libraries to have had their Python 3 bugs shaken out.
Not to mention, all the "convenience" libs that are easy to implement but would take forever to make them all yourself. Examples like jaro distance, jaro-winkel distance, etc.
Unrelatedly, I wish Nim was more popular and I wish they concentrated on math more. Having a general purpose fast language that you can prototype in is amazing. Although you can't really blame such a small community for not focusing on a niche.
Julia might be great eventually, and it's really fun to write in and much easier to implement new things in compared to Cython, but not so great right now for my purposes. You end up importing the Julia library from Python and it takes a month to JIT compile some things. Also, string processing is not nearly as easy as it is in Python.
The problem for scientific users seems to be to achieve enough consensus about which language to use as an "obvious choice" that a bunch of other scientific users make packages.
Since we are talking about "fast" languages like Nim and Julia, I wonder why R and Python have done so well with the scientific audience but C++, Java, C# and the like never developed anything quite comparable. Or did they and I just didn't notice?
Still, nice that Python will have it.
Seriously, why does it matter? Is Python 3 really a "scripting language"? I don't think so. It's fairly close to, for example, C#, that has had these particular concepts for a while now. As have many other languages. Again, I don't think it matters that much who was "first".
"It's exciting that Python is embracing these features."
That's really what you're trying to hit home.
This is the ACM Queue paper about async/await in Dart (iirc according to Erik Meijer the cleanest implementation of the concept so far).
def my_generator():
with something() as bar:
for x in baz():
yield bar(x) # context manager will call bar.__exit__() before the second time through this loop.
Python 3.4.2 (default, Dec 23 2014, 17:01:26)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from contextlib import contextmanager
>>> @contextmanager
... def printme():
... print('enter')
... yield
... print('exit')
...
>>> def my_generator():
... with printme():
... for x in range(3):
... yield x
...
>>> for x in my_generator():
... print(x)
...
enter
0
1
2
exit def lazy_lines(filename):
with open(filename) as f:
for line in f:
yield f
The Python 2/3 divide isn't that big of a deal anymore. This is not 2010. Almost all of the important libraries support Python 3; thinking especially to scientific computing. More and more distros are offering Python 3 by default. Big projects, like Red Hat's package manager, have switched to Python 3 (in Yum's case, by its replacement DNF).
It's only a matter of time before Python 2 isn't something that will be encountered in mainstream projects, only legacy systems. Which is nothing new (there's still lots of Fortran projects out there.) For them, the old version still exists. It will always exist. Should they want to make the jump to Python 3, the resources required are much less as compared to making the jump to another language.
Python 3 fixes fundamental flaws collected over the first twenty years of its inception. For those that cannot use a newer version of the language at this time, the old version is available.
Given how closely tied generators are with async/await, why do we need "async def"? Isn't the presence of "await" or "async for" or "async with" enough to mark a function as asynchronous?
See the PEP's "rationale and goals"[1] and this article[2] summarising the PEP.
[1] https://www.python.org/dev/peps/pep-0492/#rationale-and-goal... [2] https://lwn.net/Articles/643786/