A green thread implementation will thrash just as much in your hypothetical scenario.
The only advantage of green threading is the capability to do problem-specific optimized scheduling.
If you're not doing this then you don't need green threads.
The only advantage of green threading is the capability to do problem-specific optimized scheduling.
If you're not doing this then you don't need green threads.
The kernel might allocate that much, but it's just virtual memory.
The million threads risk slowing each other down with context switches, preventing any from finishing in time. An async event loop will focus on one connection at a time until `await`, and at least finish the tasks it attempts.
I haven't been in this situation, but async seems to have real advantages here. With OS threads, you don't control scheduling without costly synchronisation primitives.
You're assuming that an async event context switch is somehow vastly less costly than an OS context switch, which isn't true in the general case. (And unless you really went out of your way to make it happen then yours is the general case.)
Threads have separate stacks. Surely this costs at least a little, and adds up with the number of threads?
Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.
For async tasks, it means return, then call. For threads, it means moving to an entirely different stack, somewhere else in memory. “Loading context” is more expensive in the latter case.
> Maybe using a stack means using more cache lines than coroutines, but you'd have to be right at the edge of capacity for active requests for it to matter.
Why? Cache doesn’t only speed things up when capacity is full. Anything we have to reload from RAM will take time to load.
It only matters if you are at capacity because otherwise the active requests will be cached.
But a thread has a separate call stack. Returning and calling another function in the same stack just involves moving the stack pointer by a few bytes. Switching to another thread’s stack will invalidate much more cache, while running an async task until await will make the best use of the existing cache, and switch (much closer in RAM than in another thread) only when it makes sense, i.e. the previous task started waiting on something.
> It only matters if you are at capacity because otherwise the active requests will be cached.
CPU cache is a few megabytes. That definitely won’t hold hundreds of requests if they all transfer substantial data, even if you can handle thousands or more.
They are, and this is the entire premise green threads are built on. An OS context switch has to do a lot more including switching page tables and possibly performing a TLB flush.
(I do agree that the majority of python projects will get by just fine with regular threads though)
Isn’t the main advantage simply that you avoid making an absurd number of costly system calls?