2) Can uvloop be used with frameworks like flask or django?
3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?
2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2.
3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there are so many asyncio DB drivers and other modules now!
Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect).
If you are keen, this has the potential to be the killer application for Python 3.
This was/is the big issue with Tornado, IMO (and Tornado has been around for ages in framework time). Tornado is only async if the entire call stack all the way down to the http socket is async, using callbacks instead of returning values. This means that any 3rd party client library you use has to be completely written asynchronously, and none are in python. So you end up with a lot tedious work re-implementing http client libraries for Twilio or Stripe or whatever you're using.
I'm curious to see where asyncio goes in python, but I'm a bit skeptical after seeing how much of a pain it was to use Tornado on a large web app. In the meantime I'll be using Gevent + Flask, which isn't perfect since it adds some magic & complexity but has the huge upside of letting you keep using all the libraries you're used to.
The main benefit of Node as I see it is that the entire Node community uses the same IO Loop whereas Python's community is fragmented between normal sync code and multiple different IO Loops (asyncio will probably help with this).
Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database interaction until it's too late.
Oh and please please choose your name so that it doesn't conflict on Google search. http://www.spinframework.org
[1] http://initd.org/psycopg/docs/advanced.html#async-support vs https://github.com/chtd/psycopg2cffi
I strongly advise against this. One of the reasons Flask is so attractive is the fact that it does not enforce any database on you.
Thanks to its decoupled design you can use it purely as a routing library, which is great! Letting the framework decide something important as the database is a bad idea. [1]
[1] https://blog.8thlight.com/uncle-bob/2012/08/13/the-clean-arc...
what frequently happens is a web framework without a thought for any kind of DB interaction (or as you put it... a routing library). In things like an async web framework, that could leave users hanging. For example, psycopg vs psycopg2 vs psycogreen vs psycopg2-cffi . Tell me which one to use and benchmark it.
I agree, its a fine line. And we can keep going back and forth whether a framework should "recommend" or not recommend. But in case like this - I think there will NOT be a lot of libraries that will be compliant with the async usecase. I would hope that this framework will recommend... but not ship "batteries included".
Most database libraries don't play very well with non blocking code. In fact nodejs dB libraries were specifically designed for this.
Building async frameworks is not trivial - http://initd.org/psycopg/docs/advanced.html#async-support
Indeed, we want to make it very easy to use, especially a clean/discoverable API, simple debugging, clear errors, etc. Which is the stuff async frameworks are not good at, and an incredible added value.
It's a lot more work we initially thought. When you are doing async, you now think, not with a sequence of actions ordered in time, but with events. So for your framework to be usable, you must provide hooks to:
- do something at init
- register a component
- do something when a component is registered
- do something once it's ready
- do something when there is an error
- do something when it shuts down
But, we learned while coding the project that, with asyncio, exceptions in coroutine don't break the event look (except KeyboardInterrupt which is a weird hybrid) while exceptions from the loop do break it.
Plus you have to make a nice setup, which auto starts the event loop with good default so that people don't have to think about it for simple apps. But it must be overridable, and handle the case where your framework is embeded inside an already started event loop, and make it easy to do so.
It's one of the strong point of gevent: you don't have to think about it. With asycio/twisted, you have the benefit a explicit process switch and parallelism, but you need to pay the price with verbosity and complexity. We try to create a balance, and it's turns out to be harder than expected.
Then you have to make clear error reporting, especially iron the implementation mixing Task, Futures, coroutine functions and coroutines. Provide helpers so that common scheduling is done easily...
And you haven't even talked about HTTP yet. This is just proper asyncio management. This is why nobody made a killer framework yet : it's a looooooot of work, it's hard, and it's very easy to get it wrong. Doing like <sync framework> but async doesn't cut it.
- It's verbose compared to flask.
- It reinvent the wheel, while flask uses great components such as werkzeug.
- If you want to make a component, it's complex.
- It doesn't come battery included for the Web like Django, just the bare minimum.
- It misses the opportunity to provide task queues, RPC or PUB/SUB which are key components to any modern stacks are made easily possible by having persistent connections.
- It ignores async/await potential of unifying threads/process/asyncio and don't allow easy multi-cpu.