Gunicorn | Hacker News Reader

Gunicorn

gunicorn.org

208 pointsbryanrasmussen171 comments

I wish Gunicorn with Gevent had won the “async vs. coroutine Python” battle. The Developer UX is so much better with coroutines and just as scalable as asyncio (async/await).

Many people misunderstand asyncio thinking it will make your code faster, when what it does do is make it more scalable (if you have allot of IO for example). But at the expense of developer UX. Mixing async and sync code within Python is a nightmare. I am completely unconvinced by the move to asyncio everywhere in Python.

I have been using Gunicorn with Gevent for nearly 10 years and never had a problem with it.

Languages like JavaScript that have been async from the start work well as you (mostly) only have an async api.

https://docs.gunicorn.org/en/latest/design.html#async-worker...

http://www.gevent.org/

it kinda has - we use gunicorn with fastapi using the uvicorn asgi workers

https://fastapi.tiangolo.com/deployment/server-workers/

Green threads (gunicorn/gevent) allows you to write Python the way you always have and gain many of the benefits of async.

Uvicorn and ASGI do not. For those you need to handle async explictly.

i hear you - however the framework captures most of the abstraction entirely. In addition, most of the bottlenecks are actually i/o (db queries, etc) where its entirely worth doing it explicitly versus magic.

in a general programming language sense...ur right. But from the POV of what gunicorn/fastapi are actually used for - web frameworks - its actually quite good.

The list of terms in that sentence: Gunicorn Fastapi Uvicorn Asgi

I suppose also nginx docker?

Edit: the grandparent comment has now been updated to clarify, so my comment here is not relevant anymore.

ASGI is similar to WSGI but for async oriented web servers. The person you respond to is saying that they use Uvicorn via the ASGI interface.

See https://asgi.readthedocs.io/en/latest/.

The list of letters in that sentence: a c d e f g h i k l m n o p r s t u v x

I suppose also newlines?

I've always been a bit confused by this deployment model.

gunicorn bills itself as "Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model."

uvicorn's one-liner "Uvicorn is a lightning-fast ASGI server implementation, using uvloop and httptools."

So it seems redundant at first to be running both, but what the gunicorn+uvicorn-worker-process pattern really does is throw away the "WSGI HTTP Server" part of gunicorn and just use the "pre-fork worker" process management part, with the underlying processes actually being ASGI.

If you run in a different kind of process-management environment like k8s, does running gunicorn+uvicorn give you anything else?

Gunicorn fundamentally runs everything as a worker process.

You have sync, gthread, gevent, evenlet, tornado (tornado was also a popular server). UvicornWorker is another type of gunicorn worker... incidentally authored by the same guys who built uvicorn.

Nothing strange at all.

But ur question is genuine - but it is orthogonal to the uvicorn part of this answer. Fundamentally, in a k8s model...why bother with gunicorn?

The answer: I don't know. I do love the control that gunicorn gives me...so I'd probably use it even if I was using a single worker.

However there is a complex answer with Threads vs cores vs pods that I'm unqualified to give.

I have been using the async/await syntax in C# for more than ten years and never had a problem with it.

What is your point?

Most of Pythons standard library is sync, the minute you want to use a sync api from within async/await code it becomes a nightmare to manage.

If you are only using async library’s then async/await works perfectly.

It's straightforward to call sync functions from async ones. Are you referring to the fact that those sync functions could block and cause the async code to perform worse? I admit it's a few more lines of code to write a script that runs async code.

I've worked on several projects recently where async was used throughout and haven't had issues. There are many async libraries now, like ones to utilize GCP services, make HTTP calls, connect to Redis, etc.

Yes, I mean blocking sync calls from async code.

Sounds like you're referring to the "coloured functions" "problem", but it's the other way round. Calling a sync API from an async function is no different to calling it from a sync one. Calling an async function from a sync one requires that you make the caller async.

Personally, I like that. An asynchronous function is different to a synchronous one, and if you call an asynchronous function the caller is asynchronous. The "colouring" just reflects the realities of the problem domain. I'm not keen on libraries that monkeypatch the world to hide that fact, as gevent does.

However, some people prefer those libraries. Which is fine. They both exist, use what you like. There's no need to have wars about it.

> However, some people prefer those libraries. Which is fine. They both exist, use what you like. There's no need to have wars about it.

This results in ecosystem fragmentation though, which means that more effort is spent developing and maintaining different kind of solutions to the same problem, only because of their different developer experience.

On the contrary, look at Go, where it was decided from the beginning (i.e. stdlib) that we're going with coroutines.

It's a bit like (I'm exaggerating) arguing for what should be the default coding style, where a built-in/blessed formatter would make this a non-issue. gofmt is a nice example of this.

Also, honest question: AFAICT Python was more of the "there's 1 preferred way to do things". Doesn't arguing between async and coroutines goes against this mantra?

> Calling a sync API from an async function is no different to calling it from a sync one.

That's only true syntactically. If your sync function actually does any IO, it will block your async worker thread. Said differently it just synchronized your async code.

> the minute you want to use a sync api from within async/await code it becomes a nightmare to manage

What's wrong with sending the sync code to a thread pool? My CPU-, disk- and database-intensive application is filled with calls like:

   result = await asyncio.to_thread(long_function())

...and its resource usage and performance are great.

His point seems to be that mixing async and sync code in Python is a nightmare. I'm not sure how the fact that you do not observe the same issue in C# (a totally different language and runtime) is relevant?

And one if the very relevant differences is that C# is task-based while Python is coroutine-based.

And because C# is statically typed, it can reliably warn on missing awaits (a common issue in the also-task-based javascript, though less problematic than it is in Python).

I'm calling parent out on their weak argument by copying it. Quote:

> I have been using Gunicorn with Gevent for nearly 10 years and never had a problem with it.

Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)

Async/await in JavaScript is, a very welcome, syntactic sugar over a callback api.

The point is that almost all JavaScript library’s have always been async, traditionally with callback based apis (now async/await). You never mixed async and sync code.

That's reason why I think the push to follow event-driven (reactive) approaches in ecosystems like Java is doomed to fail (okay, that may be a bit harsh) because there is lot of existing blocking code. It will take a long time until all of the old libraries catch up. It kind of splits the whole ecosystem into half. Specifically in Java I think it's fine we are relying on blocking APIs (and threads). For most stuff it scales properly, doesn't feel unnatural and is easier to understand. I say this as a big fan of JavaScript, I just don't see the approach being introduced after the fact as a good idea.

Even better is to just use threads. Linux can handle a million threads just fine, so even if 99.99% of your clients are idle, threading won't be the bottleneck.

It's not the time when most of your threads are idle that's the problem; it's when you've accepted a million incoming socket connections, given one connection to each thread and now you're thrashing wildly trying to get any useful work done as your clients timeout, reconnect and repeat.

A green thread implementation will thrash just as much in your hypothetical scenario.

The only advantage of green threading is the capability to do problem-specific optimized scheduling.

If you're not doing this then you don't need green threads.

> The only advantage of green threading is the capability to do problem-specific optimized scheduling.

Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?

This article does a pretty good job of explaining how green events work. It seems that each of the threads will attempt to accept any/all incoming requests.

https://rachelbythebay.com/w/2020/03/07/costly/

that's an implementation detail, mostly because Linux has taken too long to copy WaitForMultipleObjects https://stackoverflow.com/questions/13913882/what-is-the-equ...

You are also optimize the scheduling for real threads, for example by increasing sched_latency_ns and sched_min_granularity_ns.

> A green thread implementation will thrash just as much in your hypothetical scenario.

The million threads risk slowing each other down with context switches, preventing any from finishing in time. An async event loop will focus on one connection at a time until `await`, and at least finish the tasks it attempts.

I haven't been in this situation, but async seems to have real advantages here. With OS threads, you don't control scheduling without costly synchronisation primitives.

A context switch happens when a thread blocks on I/O. An async event loop will context switch when 'await' happens. (Which is, also, usually when it wants to wait for I/O.)

You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)

IME async "context switches" are indeed vastly less costly, and I don't see why it wouldn't be. And I think calling it context switch is reading too much into it, when it's really just the program continuing on to operate on different data.

IIRC, coroutines are split internally into function calls, so switching to a task is actually just a function call from the event loop, in the same call stack.

Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?

It does cost a little RAM, but stack size is rarely the bottleneck, especially when using a language like Python.

But we have to switch from one stack to the other. If we do that more than necessary on thousands of threads, I imagine the cache will be used less efficiently. The problem is runtime, not RAM usage.

In either case you need to load the context for other requests. It isn't like you need to load the whole stack either.

Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.

> In either case you need to load the context for other requests.

For async tasks, it means return, then call. For threads, it means moving to an entirely different stack, somewhere else in memory. “Loading context” is more expensive in the latter case.

> Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.

Why? Cache doesn’t only speed things up when capacity is full. Anything we have to reload from RAM will take time to load.

To do anything with a request you are going to have a context which will function just like the stack of a thread. The only question is how many cache lines each takes.

It only matters if you are at capacity because otherwise the active requests will be cached.

> To do anything with a request you are going to have a context which will function just like the stack of a thread.

But a thread has a separate call stack. Returning and calling another function in the same stack just involves moving the stack pointer by a few bytes. Switching to another thread’s stack will invalidate much more cache, while running an async task until await will make the best use of the existing cache, and switch (much closer in RAM than in another thread) only when it makes sense, i.e. the previous task started waiting on something.

> It only matters if you are at capacity because otherwise the active requests will be cached.

CPU cache is a few megabytes. That definitely won’t hold hundreds of requests if they all transfer substantial data, even if you can handle thousands or more.

> You're assuming that an async event context switch is somehow vastly less costly than an OS context switch

They are, and this is the entire premise green threads are built on. An OS context switch has to do a lot more including switching page tables and possibly performing a TLB flush.

(I do agree that the majority of python projects will get by just fine with regular threads though)

Agree that that kernel can handle many sleeping threads fine. The main problem with a million threads is that they all take up some fixed memory for stack. 1 MB/stack as a rough estimate. Go’s green threads use some kind of dynamic stack implementation to solve this problem. Dynamic stack requires a language specific implementation, for ex, with C pointers it would be difficult. Depending on how python C modules work, a similar solution would be needed to just offload to the kernel.

Your threads won't be using 1MB per stack unless you are running deeply recursive algorithms, but then you'll be using lots of CPU.

The kernel might allocate that much, but it's just virtual memory.

Wait, but JavaScript only had async/await added recently too, with ES6 ? (Or was that "just" about an easier way to use these features?)!

EDIT: Argh, double posting, did I hit "back" by mistake ?

They are effectively sugar on JS promises which are effectively sugar around async event callbacks.

This is why node rose so quickly in popularity as a server. The async IO handling could be leveraged with callbacks to scale low number of processes to handle large number of requests. Plus easy to learn and lots of people knew some JS.

JS always had an event loop (in browser so async events work) and in nodeJS it can be used for async IO. So JS has basically always handled async of sorts (although that has improved significantly).

Yesterday a friend just sent me a great post on tuning gevent with gunicorn: https://eng.lyft.com/gevent-part-3-performance-e64303fa102b

gevent workers are best suited when your app spends a lot of time waiting on I/O (as you note), and they recommend setting `--worker-connections` to the ratio of {network IO time}/{CPU time}. If you set that too high, then it can result in higher latency for requests.

>I am completely unconvinced by the move to asyncio everywhere in Python

Me too, now. I've just been bitten by asyncio and it wasn't nice. Running many concurrent tasks a few of which are websockets, the state of Python websocket libraries that work with asyncio is terrible. The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks. It's incredibly hard to debug and exceptions don't get raised properly to outside an event loop, silently making your program run in a zombie state. Even stack traces instantly become a mess when you add async. Bringing something like pandas to an environment like this is probably an immense headache.

Sounds like you need some structured concurrency. Check out Trio or AnyIO.

https://trio.readthedocs.io/

It's too bad we got asyncio as the standard library module, though some SC concepts are going to make it into the library in future Python versions.

Small plug for Kotlin as well! Yes, there's new concepts to learn with structured concurrency, but I've been mostly enjoying how quickly it gets out of your way.

once properly configured :)

How would Trio help with this use-case?

Structured concurrency gives you exceptions that bubble up the stack like a normal program does, and it cancels other tasks along the way so you don't have zombies. Debugging is thusly easier as well since you keep tasks into a scope (nursery, cancel scope).

> The sockets would very silently crash until I added asyncio.sleep(0) between the more expensive CPU tasks.

That is exactly the problem with asyncio, the minute you want to do something a little more cpu heavy it blocks. Although admittedly, you would probably see the same problem with gevent, and have to use the same sleep(0) trick.

Point is for good scalability you need both traditional threads and either coroutines/asyncio to work together.

> That is exactly the problem with asyncio, the minute you want to do something a little more cpu heavy it blocks.

Isn't that by design? Sending CPU-heavy tasks to the default executor with asyncio.to_thread() works perfectly for me.

Or maybe this doesn't scale in your use-case, because of the thread count limit? (Although in that case, I don't think you'd be better off without asyncio.)

Won’t CPU-heavy tasks still make everything choke because of the GIL? You want to send CPU-heavy tasks to a different process instead of a different thread, don’t you?

It depends what you mean by "choke". I think the parents consider "choke" to mean "will focus on one long-running task at the expense of any other task that needs to run". The GIL means that multiple threads won't be able to run python code in parallel, but the design should fairly share the available processing time between all threads that want to run.

In my case, I'm lucky to be using computation libraries that release the GIL.

In my case no, because I heavily use numpy, PyTorch, and other libraries that use C++ extensions and release the GIL during long operations. I easily use 24 CPU cores fully by submitting to one thread pool.

If you use pure Python for long tasks, you're right that a ProcessPoolExecutor would be better.

Maybe the IO part of asyncio is confusing people...

The problem here is that you forgot that asyncio gives you cooperative multitasking. That means you need to hand of control periodically whenever you're doing anything CPU intensive or you end up starving everything else. That happens when you do things like an I/O operation or sleep. Either that, or you hand off the CPU intensive stuff to something on a seperate thread.

Now, I'm used to this stuff from doing GUI programming on RISC OS back in the day, and got to know instinctively when I should be using WIMP_PollIdle there. For asyncio at work, I've some wrappers around range(), iter(), enumerate(), &c., that will periodically hand off control with a zero length sleep occasionally in CPU-bound code.

I have also often wondered the same thing. I have been running wild with gevent/gunicorn for years and have struggled to adopt native async that is part of the core lib.

A pet project I would like to get started on would utilize gevent and a dynamic pool of processes to recreate an actor model environment on top of Python. That is - I hope to engineer around the GIL by distributing work to distinct processes, but at the end of the day the user just interacts with one "runtime". Trying to improve my academic knowledge of scheduling algorithms and bin-packing techniques.

I have some client projects that would greatly benefit from a move to Erlang/Elixir, but the cost/time/risk of forklifting the entire project to a new runtime is too high. I've used dramatiq.io which is advertised as an actor system, but it really isn't (actors do not have persistent state, no semantics for at-most-one, etc) I think there is a void to be filled here.

Ostensibly the underlying concurrency method is not important here, and could be stubbed out, but I will definitely get it started with gevent.

What you lament is common in parts of Rust as well - especially in web-server code. Also making inroads into embedded.

Seems like it would be a pretty natural fit in many embedded scenarios— particularly where your entire program is basically just a bunch of threads waiting for things to happen.

FreeRTOS is that anyway, except it's longjmping between heap- or statically-allocated stacks, and you have to unblock worker threads by poking them from ISRs. All of this is workable but I could definitely imagine it being more sane with first-class language support for the fundamental cooperative multitasking model that's at work.

As a minor counterpoint, the recently announced Hubris OS, implemented in Rust, is entirely synchronous.

http://cliffle.com/blog/on-hubris-and-humility/

https://oxide.computer/blog/hubris-and-humility

I'd love to see where this goes! I don't have an immediate use case since my current projects aren't complex enough to warrant an OS. Would love to have that in the toolkit.

Hubris is _very_ different, however. Each task is fully static in terms of its resources allocation, and multitasking is essentially cooperative. Now, I'm willing to be corrected on this, but a lot about it reminds me of TinyOS written with larger systems in mind.

I already write async Python on embedded for my pet projects. It actually makes a lot of sense -- with events from GPIO etc.

Almost no one really understands asyncio (I pretty much don't), but syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async".

Once in a while someone says that "yes, you can do logic programming in PHP or OOP in Haskell". Maybe, but syntax is more than a nudge, it's a finger on the scales.

> Almost no one really understands asyncio

I'd very much like a source for that, because I don't feel like it's very hard to understand.

> syntax like "async def" and "async with" makes you feel like you do, and stuff like FastAPI encourages you to type "async"

FastAPI's docs tell you to use "def" if you don't know what you're doing. [1] says "If you just don't know, use normal def."

It's also pretty simple, I think. If your function blocks, use "def", and let FastAPI run it in a thread pool to avoid blocking the event loop. If it doesn't block, use "async", and take advantage of concurrent execution without needing a new thread.

[1] https://fastapi.tiangolo.com/async/

Not that this validates the claim, but this was a pretty influential post by the author of Flask circa 2016:

https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...

I agree that it's kind of a mess, but it's getting better with simpler APIs (like asyncio.to_thread()), and in practice you can just use the primitives you want and ignore the rest.

I haven't read the post in detail though, thanks for the link! Seems interesting.

FastAPI's docs tell you to use "def," yet the most basic Hello World example in the tutorial uses "async def." In fact most, if not all, of the examples in the tutorial use "async def." So I don't think it's surprising if developers new to FastAPI tend to use async by default.

> FastAPI's docs tell you to use "def," yet the most basic Hello World example in the tutorial uses "async def."

Because that's the correct way. print("hello, world!") will block for a very short time, so it's not worth sending to a thread pool.

> In fact most, if not all, of the examples in the tutorial use "async def."

That's again because they show the correct way to use async.

Using async correctly, when it applies is preferable to a synchronous function. The docs encourage you to do it if you know how, and if your use case fits an async function.

> So I don't think it's surprising if developers new to FastAPI tend to use async by default.

It is to me, because the docs are clear:

- if you know your function can be made async properly, make it so,

- if your function blocks, use "def",

- if you don't know enough to choose, use "def".

People who complain that the event loop blocks too often simply haven't read the docs well enough.

From using FastAPI and answering some of the issues the main problem is people do not understand that python async is cooperative multi tasking and not preemptive.

There are trade offs for either but I think a lot of people come to python and don't understand event loops and how they work get bitten by it.

I just use FastAPI as a Flask replacement with Pydantic bolted on. Works out great for my use cases.

The downside is that monkey patching the world with gevent was hard to pull off and even harder to verify. I was running a large Django installation on gunicorn+gevent for some years and I saw almost no difference in throughput or response time when I made the switch to sync workers. I was glad once I could remove all the monkey patches which really did nothing for my use case.

To this day I am not sure how anyone is getting better performance from using async over sync for something like Django. If you run 1000 sync gunicorn processes, I'm sure the OS scheduler is doing a better job in assigning them CPU time slices than what you could ever do with gevent.

There is a place for gevent/asyncio, for example if you want to fire off some http requests or DB calls in parallel but you'll have to do it manually no matter what gunicorn worker you run on.

Agree on the hackish nature of the monkey patching. My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching. Or at least a lower level api was exposed allowing different concurrency/coroutine implementations for the same apis.

I have used Gevent extensively in two places; long running eventsource/sse Django views (it’s just implicit due to the Gunicorn worker) and views with hundreds of db and http api calls that are made concurrently before returning a response. Both worked well from my experience, the latter needed coroutines to be spun up for each api/db call.

For a while gevent was my go to for projects needing some kind of pub/sub service (different telemetry and device management applications). Worked great in that case but then Go came along, it works even better for those cases.

> My wish is that Gevent had been adopted into core so that it “just worked” with no monkey patching

The problem with Gevent is no matter how clean the stack switching, the major liability is other code sharing that stack -- the complete universe of native Python extensions is huge, and some of it most certainly makes assumptions about the stack.

I've used Gevent/greenlet for years and never encountered issues, but it's important not to be under any illusions about the kind of hellish crashes that might be lurking just around the corner as more third party code gets added to a project.

The bottleneck is always. The. Database.

Unless you're working on a site that's been specifically architected to not use databases. You're posting this comment on one now; HN stores everything in RAM and periodically journals snapshots out to disk so they survive reboots. Last I heard the whole site runs on one box.

> specifically architected to not use databases. You're posting this comment on one now; HN stores everything in RAM and periodically journals snapshots out to disk so they survive reboots

Yep! And for those that didn't know, this strategy on HN and other paulg projects is actually what inspired Redis! See this antirez tweet https://twitter.com/antirez/status/1110468354542919681 and https://news.ycombinator.com/item?id=19498379

Fascinating! Do you have a link to an article that explains that in more detail?

I've used gevent for years and I've benefited from being able to fire many http requests using requests and many postgresql sql queries in parallel from a django view, waiting for all of them to complete and generating a response. It was easy to do and the code was easy to write.

For the simple case, the throughput is not improved over sync workers. The scalability does improve. Basically, your code can magically cheaply wait for thousands of sockets to get ready, but it can't magically process more requests per second.

> The Developer UX is so much better with coroutines and just as scalable as asyncio (async/await).

The sad thing is, this was all known because Twisted had been around for years before they made the decision to add async/await. Deferreds, waiting for deferreds, callbacks, deferred semaphores [1]. Not to mention the bifurcated state of libraries, a few supporting Twisted and most not supporting it. And then, when gevent/greenlet/gunicorn came around it was like a breath of fresh air. I remember the day I replaced all the Twisted stuff with a monkeypatch snippet + gevent with a good 20% of code reduction with all the boilerplate gone. And then, Python 3 rolled around and it all came back in the form of async/await.

By that point I moved on to a different ecosystem (BEAM VM Elixir/Erlang) so haven't followed much what was happening. But just point out, you're not the only one who noticed it. I remember commenting on it some years ago as well [2]

[1] https://twistedmatrix.com/documents/current/api/twisted.inte...

[2] https://news.ycombinator.com/item?id=12831271

Ditto. I started out on Twisted back in the day and stayed with it for 6 years all they way up until they introduced deferreds combined with yield/generators (the same technique which is the heart of today's asyncio).

It was great at the time but ultimately tiresome. It introduced framework specific incantations all over the place and any framework using I/O had to be Twisted-aware in order to use it straight-forwardly. The exact same complaints I have today of asycnio.

I went to gevent 10 years ago and have never regretted it. If you have some reasonable knowledge of threading and synchronization then it is a really nice developer experience.

All old is new again, in different context. Windows 3.X programming was famously painful because it didn't have per-emptive processes. Each program had to yield the control. Windows API was just cooperative async messaging.

If you want to scale massively with async, do it Erlang way.

JavaScript didn’t not have any async or parallelization keywords at all. Essentially it’s just passing a delegate that is called later. This was stabdardized into Promises and all the async/await is another wrapper around the promises.

It works well, because in the end, it’s just passing a function that should be called when the process finished.

It was probably over 10 years ago now that I had my hallway moment with Guido van Rossum (creator of Python) at a PyCon. I gave him what was probably an incomprehensible pitch about the shortcomings of generators as coroutines for concurrent network server programming and why I thought stackless python's approach was what Python should adopt as its primary approach to concurrency.

Guido listened attentively despite my rambling being essentially a poor rehashing of a proposal rejected 10 years prior! I was too starstruck to remember a concrete thing he replied with, but I remember it being kind and detailed. Was an incredibly formative experience for me about how those in a position of authority and privilege should behave.

I doubt I was "right" in any universal sense. Green threads make FFI difficult and costly, and cheap and easy FFI has always been one of Python's most compelling features.

I did switch to being a Go programmer as soon as I could find a job using it and haven't looked back. I don't think there's one right abstraction for concurrency, but I think I could live with Go's for the rest of my career and be happy.

All of this to say: +1 to gevent.

Completely agree on Gevent. The UX was outstanding.

In general, I personally don't love the direction Python 3 has taken with asyncio, type hints, and more and more syntactic sugar. Meanwhile, no real improvements to packaging or to performance of pure Python code. But I'm just a bystander and Python is a big ship to steer, so I get it.

Languages like JavaScript that have been async from the start work well

This is one of several reasons I've enjoyed gradually switching over to Go from Python. Concurrency is more or less a pleasure in Go. It equals the great UX of Gevent (i.e. Goroutines) but baked into the language and multiplexed onto many real threads (versus only one thread in Gevent).

Gunicorn is one of my favorite app servers. I've been using Gunicorn in production on a bunch of projects (any Flask / Django project I happen to work with) for around 8 years now. It's rock solid and extremely predictable.

I also use Gunicorn in development too, it supports code reloading and also lets you enable Flask's debug middleware so you can use the interactive debugger. This is all configured in my Build a SAAS App with Flask course, a code example is here: https://github.com/nickjj/build-a-saas-app-with-flask.

A Django example is here: https://github.com/nickjj/docker-django-example

+1 here, we've been using Gunicorn in production for ~2 years now, once it was set-up I never had to think about it

This post would have benefited from a longer title. As it was I was disappointed to not see a picture of a horse with a gun coming out of its forehead.

For anyone similarly disappointed here is an example: https://www.deviantart.com/kirawra/art/gUnicorn-Adopt-OPEN-8...

Only $120. Go adopt it.

One main reason I switched away from Python was all the dependencies needed to run a performant webapp. And then throw async into the mix and you start wondering about uvicorn instead or do I need gevent?

Go really simplified these decisions and basically all by using the std lib.

I haven't switched away from Python (yet). But I agree that Go does look very compelling, and is basically the only performant and mainstream implementation of green threads.

Just for discussion would Erlang/Elixir not count as another performant implementation of green threads or would you not consider them mainstream enough?

Erlang has been scaling massively in the telecommunications industry for a very long time now even if it's not as popular as other languages.

Erlang's green threads are considered lightweight processes. They only communicate through message passing.

When selling Erlang, don't focus on "telecommunications industry" but sell based on Whatsapp: http://highscalability.com/blog/2022/1/3/designing-whatsapp....

More: https://www.erlang-solutions.com/blog/which-companies-are-us...

I started the switch last summer. Haven't looked back since. Python gives the veneer of simplicity (ie the Flask hello world script, 5 lines of code approx?). On top of that the 5 different ways of doing x, or just wanting concurrency. Quickly Python becomes verbose. My current webapp is one executable that uses the standard net/http server and the autocert package for Letsencrypt.

I agree that the Go deployment model is generally a dream compared to Python. However, the other part of the equation is having access to mature and complete frameworks like Django (a high quality package blending best practices and common patterns), for which I'm not sure there's an equivalent (yet) in Go.

It is interesting to me that after so many years, Django (and especially its ORM) still stands out as one of the most ergonomic, robust and reliable web frameworks you can find. Enough so that people stick with Python despite every other factor suggesting another language would be preferable.

Makes me curious, is it that hard to clone its features into other languages? Is there something specific about Python that makes aspects of it possible there while not being possible in other languages? If so, is it a true case where dynamic languages win over static languages because certain things just can't be established satisfactorily in a statically typed language?

I have tried many WSGI servers for serving Python web applications (using both Django and Flask). Gunicorn may not be the fastest or have the most features, but its super solid and the documentation is superb. Everything else that i tried hat some sort of quirks und hickups that made it unsuitable for production environments.

100% this. I started off working with uwsgi and I had problem after problem while gunicorn has always been rock solid for me.

In hindsight I wonder if it’s just experience in the experience that has led to this differing experience or if the products are actually meaningfully different in their configuration and behavior that this is consistent for others.

Thanks for sharing, this made me feel better that I had so many issues with alternatives.

For asyncio side of things there is also Uvicorn (based on libuv/uvloop) https://www.uvicorn.org/

Or Hypercorn https://gitlab.com/pgjones/hypercorn if you want HTTP/1, HTTP/2, and HTTP/3.

Note that Uvicorn still recommends to use Gunicorn for production use, with the Uvicorn worker class implemented for Gunicorn.

https://www.uvicorn.org/#running-with-gunicorn

I've been dealing with Python for 8 years at 2 big tech companies in the Bay Area.

The main advantage in Python is that it's easy to learn. It just doesn't scale. Once things start taking off, there's always a migration to something more scalable like Go or Kotlin.

Gunicorn is probably the "best" tool for making a Python based website or API for Python users.

Can we get a source on that? I've successfully used guniron for APIs getting hundreds of thousands to millions of requests a second.

I've also found it's best to question anytime someone suggests a technology "doesn't scale."

I don’t like GP’s phrasing, but the post describes a real phenomenon. Beyond the most trivial examples, concurrency and parallelism in Python are a massive pain, as many examples here attest.

Unless you have a very compelling reason to stick with Python (e.g., dependence on its excellent data science libraries), there are almost always better choices if your project has to coordinate many things happening at once.

You can tell how Gunicorn is old by looking at example showing only 4 workers :)

4 highly skilled workers. Imagine how much 4096 workers would cost?

4TB.

> You can tell how Gunicorn is old by looking at example showing only 4 workers :)

I've seen folks run a $90,000 / month business through their SAAS app using gunicorn on a 2 CPU core box (4 workers).

Nothing wrong with being efficient!

4 workers is the sweet spot for most wsgi applications. Typically you will scale above that by running additional complete gunicorn instances (managed by some kind of external scaling/load balancing thing). If it makes sense for your application to run more than 4 workers in one gunicorn instance (ie. you have long running requests that only wait on something) then gunicorn (and in fact, "normal" wsgi itself) probably is not the right solution for your application.

Edit: somewhat typical issue with scaling python applications is that just throwing additional workers at the problem in the same instance of the wsgi container (including separate uwsgi processes) stops being effective fairly soon and makes the problem worse.

We're running Gunicorn behind Nginx for an internal Django web application. I thought the rule of thumb was to scale workers = (Cores * 2) + 1?

In my experience it also depends on the worker type, if you are running sync you might need more workers depending on how much your database can handle.

Otherwise the number of workers might limit the total throughput quite severely.

We run single worker Uvicorn instances...we just run lots of them (behind an HAProxy sidecar).

Nice to see Gunicorn here. While it's modelled after Ruby's Unicorn, it's actually bit more capable as it supports the multi-process & multi-thread execution model.

It even supports ASGI by specifying a particular Uvicorn worker class and thus can remain one go-to Python application server. This is also the reason I am using Gunicorn for the examples in my book Deployment from Scratch.

How well is Gunicorn suited for production?

Do you need to put another server in front of it for https or has Gunicorn https handling built in?

Gunicorn does support https. They still seem to recommend using a different server in front of it.

If you put Apache or Nginx in front of it, why not leave out Gunicorn completely?

Apache has mod_wsgi, nginx not so much.

mod_wsgi is a much less flexible setup when it comes to mixing and matching different python versions

Python has an interface called WSGI (web server gateway interface), with both Apache and Nginx you need either a none standard module for that interface or to reverse proxy to a second server that does.

So same number of moving parts either way.

How is libapache2-mod-wsgi-py3 "none standard"?

I meant it’s not built in as standard when you install apache, it’s an extra thing to install (even if it runs in process).

Gunicorn is not just a static content server (actually I don’t think it can serve static content out of the box?) but a wsgi-compliant application server. You can’t leave it out because then, how would your python web backend run?

For apache, there is libapache2-mod-wsgi-py3

Seems easier to just "apt install libapache2-mod-wsgi-py3" and put a "WSGIScriptAlias" directive into your apache config than to run another piece of software?

I think it depends on how your whole stack looks. If you are running on a PAAS such as Heroku, have load balancers or are even using something like ClourFlare I don’t this it’s necessary to have something in front of Gunicorn.

I wouldn’t leave Gunicorn “naked” for want of a better word, or use it for static files though.

Here’s why according to their docs:

> Although there are many HTTP proxies available, we strongly advise that you use Nginx. If you choose another proxy server you need to make sure that it buffers slow clients when you use default Gunicorn workers. Without this buffering Gunicorn will be easily susceptible to denial-of-service attacks.

https://docs.gunicorn.org/en/stable/deploy.html

I was tempted to write "you always need to put a many-years-battle-tested specialized server in front, if you mean protection from attacks", but nowadays even these servers get so many new features and rewrites so often, that it's pretty valid to doubt their maturity.. And being written in C is always a time bomb.

But probably it's still a good idea to use them.

All the companies I ever worked for used Gunicorn to serve Python apps. You can put Nginx/Traefik/whatever in front of it for load balancing but you don't have to.

I've recently deployed an app using Nginx + Gunicorn on a VPS. I wrote a tutorial on how to replicate the setup here:

https://costapiy.com/deploy_django_project_linux_server/

I really like the clear example at the bottom of the page.

OT, but the biggest cultural change disappointment between my previous job and this has been readme.md expectations.

Previous: every project org-wide contained enough environment & setup information to get it running. Even if you didn't know anything else about it

Current: almost nothing

The initial "What magic incantations must I make?" step, before you're able to start really learning by experimenting, is invaluable.

The multi-py project has a cross platform gunicorn container that works with Intel and Arm out of the box.

https://github.com/multi-py/python-gunicorn

There's also a uvicorn one there for people who want the async features.

https://github.com/multi-py/python-uvicorn

And a container that combines both (although it's often better to just use uvicorn and add more containers to your API to scale up).

https://github.com/multi-py/python-gunicorn-uvicorn

> it's often better to just use uvicorn and add more containers to your API to scale up

Why is that?

Falcon + Gunicorn + Meinheld is incredibly under appreciated

Reading it as gun-icorn lead to extreme excitement which was crushed to find the link had nothing to do with gun unicorns...

I was really disappointed that they didn't replace the horn with a gun in the logo.

I assume it's pronounced gyoonicorn (just unicorn with a g in front) rather than gun-icorn?

I'm glad I'm not the only one wondering this. I found this Quora post (https://www.quora.com/How-do-you-pronounce-gunicorn) as well as this GitHub issue (https://github.com/benoitc/gunicorn/issues/139), and it's clear as mud now.

I've always pronounced it G-unicorn. As-in, say G first, then unicorn. Like Geeunicorn.

That's how I've heard others pronounce it too.

gunicorn and gevent developer UX is so nice that it prepared me for Go years later.

With Python you have to run a web server in front of your web server... never seemed right to me. Just use Golang, it's all self contained.

In fact, you don't. Your WSGI app is not a web server. It does not run or network by itself and doesn't speak HTTP. You need a compatible web server (e.g. Gunicorn) to do that for you. It's really not that much different from good old CGI. It's simple and flexible enough for a large number of use cases. Of course, that's not the only way to write web apps in Python. Using a library like Pyramid et al, you launch the HTTP endpoint manually from the main program, and attach various handlers/whatnot to it. All "self-contained".

I love Python for slicing and dicing data, automating tasks and building tools, but for a web server or web application it's a mess and is a bad choice.

What do you base this judgment on? I've built several web applications using Python and things worked very well and were far from being a mess.

I cannot be the only one disappointed by the lack of unicorns with guns.

Frankly at first I misread it as Gnunicorn and was just relieved not to see Richard Stallman in a fursuit.

I misread it as well, thought "it can't be, or it's a parody", re-read it, "oh, unicorn and gun?".

The disappointment....

well yeah, things sure would have been more fair https://en.wikipedia.org/wiki/The_Hunt_of_the_Unicorn

https://ih1.redbubble.net/image.1433836564.0273/raf,750x1000...

I was disappointed not to see an article about a unicorn of guns, like a Star Trek phaser pistol or some such.

I love Python. But for web projects, PHP has a massive upside over Python: You don't need to restart your application when you change the code.

I know people run monitoring scripts on their development systems that automatically restart Apache or whatever server they use every time they change a line of code. Still this is a cludge.

Development in PHP is:

    - Change a line of code
    - Alt+Tab to switch to Firefox
    - F5 and see the result

So I still enjoy PHP development more when it comes to the web. Even though PHP's namespaces are crap compared to Python's module system.

Right now I have gunicorn running with --reload.

Change a line of code, refresh/rerun client and see the result.

Live/hot reloading is a thing in various web frameworks and supported by various ecosystems, from Java over Python up to Nodejs. Funnily, development with hot reloading systems is:

   - Change a line of code
   - Alt+Tab to switch to Firefox
   - No need to press F5 because site already updated.

And so does Rails and Sinatra in Ruby land.

I have encountered very few development servers that lacked hot-reloading in decades. Many such stacks will even inject some JS or headers to automatically refresh the browser for you on detected changes.

I think OP may have an outdated or skewed experience of web-development stacks. Where PHP through apache+mod_php had "instant code refresh" back in early 2000s, and other stacks (Beans, CGI, etc) required a manual reboot everytime you'd want to check your changes in a browser. But this hasn't been true since Rails was released in 2004~5.

Ironically the most difficult one I recently encoutered was a PHP-FPM with APC, that required a "manually" restart of nginx and php-fpm in the right order after every code change. But this really is not suited as "development" server, as shown in this anecdotal issue.

> and other stacks (Beans, CGI, etc) required a manual reboot

Hey, now! Don't go lumping CGI in with those. :) CGI had the same "instant code refresh" as PHP since before PHP even existed. It even had a better "shared nothing" approach since you were guaranteed to start an entirely new process for every request.

Sorry. CGI is a bit unfair. I recall it refreshed fine with Perl, but CGI wrapping compiled languages (I did some C webdev back then) was a bit more involved. It's vague, but IIRC, that had more to do with the compiler than the CGI, indeed.

Fair. My main experience with CGI was bash, Perl, and Python so I tend to think of compiled languages in a completely different context.

I hate PHP too but you have to admit that its execution model is way simpler and more reliable than any hot reloading system. In my experience hot reload saves you a bunch of time when it's working, and then costs you the same amount of time when it randomly breaks and you spend half an hour getting confused about what the hell is going on before realising that a simple refresh fixes it.

The only really good hot reload system I've seen is Flutter's.

Yeah, I never have to manually restart my Django development server though. When I hit save in my editor, the runserver process restarts itself. At this point, I don't even know off the top of my head which part of the stack is doing the watching and restarting. It's just not something I've had to think about in like a decade.

Elixir doesn’t even blink. Make a change in a file and it’s immediately live, without even restarting the process.

It's even more fun when you upload your code directly to an FTP server and some of the customers load their page to see hundreds of lines of stack traces because when the script was executed it only saw half of the uploaded files at the same time.

the real advantage of php seems to me is not in "generic" web projects that start from scratch but the very popular domain specific frameworks that have been build on php (blogs, wikis, etc)

while python is gaining ground on such specialized areas (e.g things build on top of django) it is still very far behind in terms of adoption

> PHP has a massive upside over Python: You don't need to restart your application when you change the code.

I see this as a downside: hot-reloading is something that should be part of your developer toolbox, not a feature of the language runtime itself. Otherwise, deployments are harder because by the time you’ve completely uploaded all your updated PHP files, the server already started to execute the code in the ones uploaded so far. Also, I hope you have a caching mechanism otherwise this "read the code from disk every time you want to execute it" must be a performance nightmare.

None of this is a problem in modern PHP. In production you flip a flag to turn on operation caching. You can also tell it to preload all of your code. You can then deploy the app and then clear the cache once all the files are there.

Is there an implementation out there of hot reload that doesn't simply automate hitting the refresh key in the browser? Because if not then this vaunted 'hot-reload' feature is a joke.

Some of the JS kiddos from my previous org were pushing this crap on me from some random React module that I can't remember and it was terrible. We had to run a second server that proxied our personal dev webserver just so the thing could send window.reload()

> Is there an implementation out there of hot reload that doesn't simply automate hitting the refresh key in the browser?

Yes, there are some setups where the code reloads but the state stays there. I’ve worked like this with Clojure/ClojureScript in the past; for example if you’re working on a pop-in that appears after a click on a button, with window.reload()-based hot reloads you have to click on the button each time you reload the code, whereas with a state-preserving hot reload you don’t. It’s very useful on SPAs.

I first develop backend APIs using TDD without even opening a browser. Then I develop frontend using React/Vue with hot reload and don't even need to press F5.

Thats a solved problem, even for non web related programming. I save in vim, and autorun my binary (multiple languages) in under 1 sec, and see the result an a vim split.

For web dev, there are countless "watchers" that auto restart on a file change. I pick any other language then PHP, on any day. Its a hot mess, and even now in PHP8.X multiple things are still broken (and will probably always be), its just a sad language to work with.

Does it work like Erlang: The movie, though? I.e. change the code without losing any connection or state.

Reply on news.ycombinator.com