SuPPort: Evented Python servers | Hacker News Reader

SuPPort: Evented Python servers

paypal-engineering.com

51 pointsmhashemi12 comments

This is a nice balance of the trade offs of the various concurrency primitives in Python.

Greenlets for lightweight concurrency, threads for preemptive concurrency and processes for parallelism / isolation.

Interesting this is built using gevent since in Python 3.4 things are moving away from gevent in favor of Asyncio. The last time I checked (a few months ago) gevent is only supported on Py2.7 unless you're using a fork that has patched Py3 support which still has a number of outstanding bugs. I'm curious what the migration path is going to look like to 3.0+ since 2.7 is near EOL (2020?).

That's a pretty coarse description of the Python concurrency landscape. "things are moving away"?

For one, there was never really official Python backing for gevent and other coroutine-oriented programming models. Remember asyncore? From our standpoint, Python 2.7 continues to be the industry standard, and we'll continue to target it for the foreseeable future. Should some major internal or open-source use case come along and make it advantageous for us to target Python 3, we would approach it like any other feature request. Python 2 will not be vanishing from beneath your feet anytime soon. ;)

It's less about where Python 2.7 is going and more about where the libraries themselves are going. If you use a library which is considered deprecated by a large part of the python community, don't be surprised not to aspire new contributors.

No one expects 2.7 to go away, that doesn't mean contributors want to deal with Python 2 when 3 is available elsewhere.

I can't fault PayPal for releasing their code as open source. It's great. More companies should do it.

But indeed, I don't think it would be a good idea to wholeheartedly adopt their stack. gevent wasn't even a good idea on Python 2, and I suspect the large companies that use it have built up an armor of best practices and workarounds for its glitches, which you won't know about if you start using it.

I realize this is a day late and thus ancient history in HN time, but I figure a reply couldn't hurt. You're absolutely right that it would be unwise for the typical Python team to fully adopt the stack, especially this early in the release.

In fact the earliness of SuPPort's release serves to illustrate that the primary goal: demonstrating many best practices for building servers and clients, in any language or stack. As far as gevent goes, well, that's mostly an ecosystem-dependent turn of fate that continues to serve us well. A data point gladly share, but not the core message per se. :)

It's tendentious to describe gevent as deprecated.

Like Twisted, there are many projects that continue to choose to use gevent, and consequently (for either of those libraries) Python 2. They solve real problems, as does the codebase we extracted SuPPort from, and we hope to help these projects rather than shun them.

Additionally, asyncio doesn't provide the same user API that gevent does: automagic task switching. Whether or not you like that API is a separate issue.

Finally, even if a person has no interest in using SuPPort, we hope that it might provide her insights into problems she'll likely encounter when using asyncio (or Tornado or Twisted or eventlet or any evented system.)

I re-read my comment and found it to be unwittingly snarky, I apologize. I really like the fact you guys open sourced your stack. I just meant to warn you that 2.x-only stacks are unlikely to attract new people because, as I said, when 3.4 is out there and usable, why would someone start something fresh on 2.x?

stack looks solid, coroutine, stats, and utilities, i'm interested in giving it a spin. some interesting libraries in there i haven't seen before. even then its hard to reconcile statements like native gevent (sans monkey) with libraries like flask (thread locals) or even the std lib. one of the main benefits of gevent vs twisted or other async libs is the ability to actual reuse the python ecosystem via a handful of monkey patched modules, a position which this stack effectively disavows.

gevent has lots of benefits outside monkeypatching! That said, historically we mostly chose gevent as it seemed like the better interface to greenlet, which we really liked. For the most part, we were (and in many cases still are) trying to cater to a Python community of beginners. Greenlet-based approaches seemed easier to explain than other options, and sure enough we rarely hear complaints about development complexity, etc.

OK kind of TLDR; I just took the packages that they use from the link they had provided.

boltons==0.4.1

cffi==0.8.6

clastic==0.4.2

cryptography==0.7.2

enum34==1.0.4

faststat==0.3.1

gevent==1.0.1

greenlet==0.4.5

hyperloglog==0.0.9

lithoxyl>=0.1.0

psutil==2.2.1

py==1.4.26

pyasn1==0.1.7

pycparser==2.10

pyjks>=0.3.0.1

pyOpenSSL==0.14

pytest==2.6.4

sampro==0.1

ufork>=0.0.1

six==1.9.0

Werkzeug==0.9.4

they're so close to alphabetical; this is killing me

Reply on news.ycombinator.com