1. For multiples processes use `gunicorn` [1]. Runs your app across multiple processes without you having to touch your code much. It's the same as having the n instances of the same backend app where n being the number of CPU cores you're willing to throw at it. One backend process per core, full isolation.
2. For multiple threads use `gunicorn` + `gevent` workers [2]. Provides multiprocessing + multithreaded functionality out of the box if you have IO intensive. It's not perfect but works very well in some situations.
3. Lastly, if CPU is where you have a bottleneck, that means you have some memory to spare (even if it's not much). Throw some LRU cache or cachetools [3] over functions that return the same result or functions that do expensive I/O.
[1]: https://www.joelsleppy.com/blog/gunicorn-sync-workers/
[2]: https://www.joelsleppy.com/blog/gunicorn-async-workers-with-...
This will load up multiple processes like you say. OP loads a large dataset and gUnicorn would copy that dataset in each process. I have never figured out shared memory with gUnicorn.
Assuming you're on Linux/BSD/MacOS, sharing read-only memory is easy with Gunicorn (as opposed to actual POSIX shared memory, for which there are multiprocessing wrappers, but they're much harder to use).
To share memory in copy-on-write mode, add a call to load your dataset into something global (i.e. a global or class variable or an lru_cache of a free/class/static method) in gunicorn's "when_ready" config function[1].
This will load your dataset once on server start, before any processes are forked. After processes are forked, they'll gain access to that dataset in copy-on-write mode (this behavior is not specific to python/gunicorn; rather, it's a core behavior of fork(2)). If those processes do need to mutate the dataset, they'll only mutate their copy-on-write copies of it, so their mutations won't be visible to other parallel Gunicorn workers. In other words, if one request in a parallel=2 gunicorn mutates the dataset, a subsequent request has only a 50% likelihood of observing that mutation.
If you do need mutable shared memory, you could either check out databases/caches as other commenters have mentioned (Redislite[2] is a good way to embed Redis as a per-application cache into Python without having to run or configure a separate server at all; you can launch it in gunicorn's "when_ready" as well), or try true shared memory[3][4]
1. https://docs.gunicorn.org/en/stable/settings.html#when-ready 2. https://pypi.org/project/redislite/ 3. https://docs.python.org/3/library/multiprocessing.html#share... 4. https://docs.python.org/3/library/multiprocessing.shared_mem...
1) gunicorn or any solution with multiple processes is going to just multiply the RAM usage. Using 10-100GB of RAM per effective thread makes this sort of problem very RAM bound, to the point that it can be hard to find hardware or VM support.
2) This isn't I/O bound.
3) If your service is fundamentally just looking up data in a huge in-memory data store, adding LRU caching around that is unlikely to make much of a difference because you're a) still doing a lookup in memory, just for the cache rather than the real data, and b) you're still subject to the GIL for those cache lookups.
I've also written services like this, we only loaded ~5GB of data, but it was sufficient to be difficult to manage in a few ways like this. The GIL-ectomy will probably have a significant impact on these sorts of use cases.
We had an issue with the service I mentioned above where VMs with ~6GB RAM weren't working, because at the point that gunicorn forked there was instantaneously >10GB RAM usage because everything got copied. We had to make sure that the data file was only loaded after the daemon fork, which unfortunately limits the benefits of that fork, part of the idea is that you do all your setup before forking so that you know you've started cleanly.
Multiprocessing. The answer is to use the python multiprocessing module, or to spin up multiple processes behind wsgi or whatever.
> Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it.
Use the python multiprocessing module. If you've already written it with the multithreading module, it is a drop in replacement. Your data structure will live in shared memory and can be accessed by all processes concurrently without incurring the wrath of the GIL.
Obviously this does not fix the issue of Python just being super slow in general. It just lets you max out all your CPU cores instead of having just one core at 100% all the time.
It may leave many useful bits on the table (compared to pure multithreaded coding, like C++/pthreads) but I've still been able to get it to scale my application performance (CPU-bound, large-memory) to the number of cores of even large boxes (96+ vCPUs). IIRC the future/concurrent library was key to being productive.
20 years ago I would said different, as at the time, IronPython demonstrated a real alternative to CPython that was faster, and fully multitrhreaded (including the container classes).
I don't know how useful it is really, if you really want performance then you probably shouldn't choose Python to begin with, or you use the libraries which may not be compatible with IronPython. These days it barely takes me longer to build a simple script in C# than in Python either.
When we needed python c# interop it was better to use python.net and integrate that way. Annoying to setup but when it works you can get both to work seamlessly
Only problem is that 99% of that CPU usage is for serializing/deserializing IPC messages and total throughput would have been higher using a single process.
There are use-cases for multiprocessing. As long as data sharing between processes is insignificant, it can be quite performant. Just like using a bash-wrapper script that orchestrates a bunch of python (or other) processes.
Only partially. When you send things to a multiprocessing.Pool/concurrent.futures.ProcessPoolExecutor, they're pickled and copied. "Sending" happens when passing arguments to e.g. "multiprocessing.Pool.apply_async()", "multiprocessing.Queue.put()" or "concurrent.futures.ProcessPoolExecutor.submit()".
However, there are two other ways to share data into your multiprocessing processes:
1. Copy-on-write via fork(2). In this mode, globally-visible data structures in Python that were created before your Pool/ProcessPoolExecutor are made accessible to code in child processes for (nearly) free, with no pickling, and no copying unless they are mutated in the child process. Two caveats here, which I've discussed in other comments on this thread: mutation may occur via garbage collection even if you don't explicitly change fork-shared data in Python[1]; and fork(2) is not used by default in multiprocessing on MacOS or Windows[2].
2. Using explicit shared memory data structures provided by Multiprocessing[3][4]. These do not incur the overhead (in CPU or copied memory) that pickle-based IPC does, but they are not without complexity or cost.
Unfortunately, truly "seamless integration" is not really possible with multiprocessing, so users will have to use one or more of the above strategies according to their application needs.
1. https://news.ycombinator.com/item?id=36940118 2. https://news.ycombinator.com/item?id=36941791 3. https://docs.python.org/3/library/multiprocessing.html#share... 4. https://docs.python.org/3/library/multiprocessing.shared_mem...
This means you need to deal with stuck/dead processes. I’ve used multiprocessing extensively and once you hit a certain amount of usage, even in a pool, you just get hangs and unresponsive processes.
I’ve also written a huge amount of Cython wrapped c++ code which releases the GIL. This never hangs and I can multithread there all I want without issue.
If processes were a universal substitute for threads we wouldn't have threads. That reasoning only gets stronger when you apply python's heavy limitations, but it gets the most strength when you experience the awkwardness of multiprocessing firsthand.
if you are managing a fleet of machines and have some tasks to do on each machine, then multiprocessing is the life saver.
Haven't played with multiprocess at scale, so am genuinely interested.
My impression is that the multiprocessing module is a heroic effort, but unfortunately making the whole system work transparently across multiple OSs and architectures is a nearly insurmountable problem.
It provides a nice interface but is using multiprocessing or multi threading under the hood depending on which executioner you use:
> The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module, which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.
What if it hangs?
I've seen orphaned processes sometimes idle, sometimes busy doing god knows what. But Zombies OTOH are rarely a problem, and should be able to be dealt with easily.
Perhaps the desire of Python to be Windows compatible mitigates against some design more suitable for Unix.
Not really, unless you want to alter it. The OS uses copy on write behind the scenes for forked processes, so will use the same memory locations already loaded until/if you modify that. So parent memory isn't really duplicated.
As for any new memory allocated by each child process, that's its own.
I'll copy the tl;dr from another comment I've made nearby:
There are three main ways to share data into your multiprocessing processes:
1. By sending that data to them with IPC/pickling/copying, e.g. via "multiprocessing.Pool.apply_async()", "multiprocessing.Queue.put()" or "concurrent.futures.ProcessPoolExecutor.submit()".
2. Copy-on-write via fork(2). In this mode, globally-visible data structures in Python that were created before your Pool/ProcessPoolExecutor are made accessible to code in child processes for (nearly) free, with no pickling, and no copying unless they are mutated in the child process. Two caveats here, which I've discussed in other comments on this thread: mutation may occur via garbage collection even if you don't explicitly change fork-shared data in Python[1]; and fork(2) is not used by default in multiprocessing on MacOS or Windows[2].
3. Using explicit shared memory data structures provided by Multiprocessing[3][4].
1. https://news.ycombinator.com/item?id=36940118 2. https://news.ycombinator.com/item?id=36941791 3. https://docs.python.org/3/library/multiprocessing.html#share... 4. https://docs.python.org/3/library/multiprocessing.shared_mem...
Only if it can be immutable. So it can't be shared and changed by multiple processes as needed (with synchronization).
And even if you can have it mostly immutable, if you need to refresh it (e.g. after some time read a newer large file from disk to load into your data structure), you can't without restarting the whole server and processes.
So, it could work for this case, but it's hardly a general solution for the problem.
Even accessing objects from the shared memory would cause the reference counter to increment and the data would be copied, causing a memory usage explosion.
In Python yes. In Java you could take advantage of shared memory and get spared the overhead of SQLite.
Nowadays numba is usually a better solution for when you want to run some computationally expensive python code that itself calls numpy, etc.
For the parent commenter's use case though that wouldn't be a great solution either. In general, Python does not have an optimal way of operating on a shared data structure across OS threads and certainly not in a way that doesn't require forking the interpreter.
A lot of people here mention that sharing data is much easier with multithreading, but doing this without races is not easy.
You can't just use the values from difference threads like you would in normal code, you need to synchronize access with locks, which can be difficult to do correctly and can harm performance in a lot of cases.
I think a lot of the people who complain about the GIL are going to become acutely aware of why it was useful when they attempt to use GIL-less multithreading, and realize that removing it wasn't as great as it sounded at first!
In my experience, most problems are inherently synchronous with lots of mutable state and complex data dependencies, or inherently parallel with lots of tasks that can run independently. Problems that can be easily parallelized already work fine with multiprocessing! Problems that can't be easily parallelized are not something you can just slap some threading on to get more performance, and will require a lot of work to keep state synced!
This is just my opinion though and I'm sure there are plenty of domains that I don't have experience with that will benefit from no-GIL python!
When you have to deal with processes, there’s a lot of external factors out of your control because processes are much more visible and carry a lot of extra baggage.
Hard multithreading problems are fun. Hard multi-process problems are just tedious.
You can create a process/thread chimera with certain system calls, and get something that is in-between a thread and process if you want, which is neat but maybe not that useful.
Creating processes on Linux is actually much faster than people seem to realize. I can spawn at least a few thousand a second from a quick test of spawning bash instances.
Yeah, except afaik you pay more in context switches, sharing is more cumbersome. Also language runtime of a single process is likely working with less information, you end up using more memory on multiple language runtime instances
Frankly I'd just use Java or Go at that point and not even bother
Yes, locks like mutexes, semaphores, etc. and approaches like atomics, lockfree datastructures come into play when writing multithreaded code. There's no getting around that.
> In my experience, most problems are inherently synchronous with lots of mutable state and complex data dependencies, or inherently parallel with lots of tasks that can run independently. Problems that can be easily parallelized already work fine with multiprocessing!
This is a hot take though-- most problems that are truly embarrassingly parallel don't work as well as you'd think w/ multiprocessing. There's a ton of overhead there and when you do need synchronization steps (eg; in reductions) it can get pretty messy.
I assume mod_wsgi under apache was not the answer here due to memory constraints. That being said, why not serve from disk and use redis for a cache. This should work well unless the queries had high cardinality.
If the bulk of the data is immutable (or at least never mutated), it can be safely shared though, via shared memory.
That depends on how you're using multiprocessing. If you're using the "spawn" multiprocessing-start method (which was set to the default on MacOS a few years ago[1], unfortunately), then every process re-starts python from the beginning of your program and does indeed have its own copy of anything not explicitly shared.
However, the "fork" and "forkserver" start methods make everything available in python before your multiprocessing.Pool/Process/concurrent.futures.ProcessPoolExecutor was created accessible for "free" (really: via fork(2)'s copy-on-write semantics) in the child processes without any added memory overhead. "fork" is the default startup mode on everything other than MacOS/Windows[2].
I find that those differing defaults are responsible for a lot of FUD around memory management regarding multiprocessing (some of which can be found in these comments!); folks who are watching memory while using multiprocessing on MacOS or Windows observe massively different memory consumption behavior than folks on Linux/BSD (which includes folks validating in Docker on MacOS/Windows). There's an additional source of FUD among folks who used Python on MacOS before the default was changed from "fork" to "spawn" and who assume the prior behavior still exists when it does not.
This sometimes results in the humorously counterintuitive situation of someone testing some Python code in Docker on MacOS/Windows observing far better performance inside Docker (and its accompanying virtual machine) than they observe when running that same code natively directly on the host operating system.
If you're on MacOS (not Windows) and wish to use the "fork" or "forkserver" behaviors of multiprocessing for memory sharing, do "export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" in your shell before starting Python (modifying os.environ or calling os.setenv() in Python will not work), and then call "multiprocessing.set_start_method("fork", force=True)" in your entry point. Per the linked GitHub issue below, this can occasionally cause issues, but in my experience it does so rarely if ever.
1. https://github.com/python/cpython/issues/77906
2. https://docs.python.org/3/library/multiprocessing.html#conte...
No. This behavior is present on any Python 3.8 or greater running on MacOS, enforced via "platform == darwin" runtime check: https://github.com/python/cpython/pull/13626/files#diff-6836...
You can check the default process-start method of your Python's multiprocessing by running this command: "python -c 'import multiprocessing; print(multiprocessing.get_start_method())'"
If you're thinking about parallelizing your Python process, chances are your Python code is CPU-bound. That's when you should stop and think, is Python really the right tool for this job?
From experience, translating a Python program into C++ or Rust often gives a speed-up of around 100x, without introducing threads. Go probably has a similar level of speed-up. So while you can throw a lot of time fighting Python to get it to consume 16x the compute resources for a 10x speed-up, you could often instead spend a similar amount of time rewriting the program for a 100x speed-up with the same compute resources. And then you could parallelize your Go/Rust/C++ program for another 10x, if necessary.
Of course, this is highly dependent on what you're actually doing. Maybe your Python code isn't the bottleneck, maybe your code spends 99% of its time in datastructure operations implemented in C and you need to parallelize it. Or maybe your use-case is one where you could use pypy and get the required speed-up. I just recognize from my own experience the temptation of parallelizing some Python code because it's slow, only to find that the parallelized version isn't that much faster (my computer is just hotter and louder), and then giving in and rewriting the code in C++.
Once you've done that, then you should be familiar enough the code to know which bits are worth using multiprocessing on (i.e. the large embarrassingly parallel bits), which if they are a significant part of your code should scale near linearly.
The other thing to check is which libraries are you using (and what are your dependencies using). numpy now includes openblas (though mkl may be faster for your usecase), but sometimes you can achieve large speedups though choosing a different library, or ensuring speedups are being built.
Threads make it transparent to the OS that this memory really must be shared between compute tasks.
If you do have memory issues, calling 'gc.freeze()' right before creating your multiprocessing.Pool/Process/concurrent.futures.ProcessPoolExecutor is sufficient to mitigate refcount-related page dirtying in the vast majority of cases. In the small remaining minority of cases, 'gc.disable()' as suggested by the freeze docs[1] may help. If that still doesn't do it, or if your page-dirtying is due to actual mutations of data (not just refcounts), it may be time to reach for actual shared memory instead[2][3].
1. https://docs.python.org/3/library/gc.html#gc.freeze 2. https://docs.python.org/3/library/multiprocessing.html#share... 3. https://docs.python.org/3/library/multiprocessing.shared_mem...
The response, which isn’t technically wrong, is “unless you’re CPU bound, your application should be parallized with a WSGI. You shouldn’t be loading all that up in memory so it shouldn’t matter that you run 5 Python processes that each handle many many concurrent I/O bound requests.”
And this is kinda true… I’ve done it a lot. But it’s very inflexible. I hate programming architectures/patterns/whatnot where the answer is “no you’re doing it wrong. You shouldn’t be needing gigs of memory for your web server. Go learn task queues or whatever.” They’re not always wrong, but very regularly it’s the wrong time to worry about such “anti patterns.”
It tends to get them exceptionally mad because their concern isn't the ideal way to write the code and architect the system, they simply want to write just enough code to continue their research, and even if they did care about proper architecture, they don't have the time or interest in learning/testing a new library for every little thing. They'd rather be putting that time reading up on their field of research.
To be clear, I’m not advocating for data scientists to write production-grade webapps. But I absolutely think they should be bothered to write code that fulfills minimal requirements, is reproducible, documented, and mostly bug-free.
Reproducible, documented and bug free is fine, they care plenty about those things too, the issue is the "no you're doing it the wrong way, use this entirely different technology instead" being based almost entirely on ideological reasons.
If we take C multithreading as an example, with my superivising scientist, multithreading is fine, he's willing to put some time into learning how it works because it's valuable and has had a stable interface backed by a reliable body for a while now. But if tomorrow you came up to him and insisted that doing multithreading was wrong without a solid technical reason (eg actual bugs and an explanation of how the only way to fix it is to dump the existing code and spend a few months redesigning) you'd get shot down.
Or like the martial arts student asking the master "how do I fight a guy 100m away with a rifle?" - "don't be there".
ton of additional complexity, not worth it for many use-cases and anything on the line of "using multiple processes or threads to increase python performance" does have (or at least did have) quite a bunch of additional foot guns in python
In that context porting a very trivial ad-hoc application to Java (or C# or Rust, depending on what knowhow exist in the Team) would faster or at least not much slower to do. But it would be reliable estimable by reducing the chance for any unexpected issues, like less perf then expected.
Basically the moment "use mmap" or "use multi-processing" is a reasonable recommendation for something ad-hocish there is something rally wrong with the tools you use IMHO.
The problem why we wouldn't want to port this application to another language is 100k lines of existing code that is best written in Python and no resources to rewrite all that.
Hmm. So you're saying only languages which bury lock and mutex over shared data are appropriate to use for async parallelism over shared data? Because calling explicit lock() and releae() isn't that hard. However it does incur a function call overhead. I suppose some explicit in language support could minimise that partially.
Actually the resulting structure should of course be dumped into an RDBMS or a graph DB and served from there more readily. Doing that takes skill and time though, which often are worth applying elsewhere.
Now, you wanted it simple, but got to fight with the memory model of a language that wasn't designed with performance in mind, for programs whose focus wasn't performance.
If you load something big before forking workers, there's no CoW issue with that big structure anymore.
While I hate how verbose and inexpressive it is, Go does hit a sweet spot of fairly good performance, even multi-core, while still being GCed so it's not nearly as foreign for a native python user.
This is absolutely an expected outcome.
And that CPython's I/O isn't really the problem: some of its async event loop implementations are fairly competitive with Node.
But still ... yes.
Javascript has benefited from two decades of intensive, well-funded work by the best people in the business, with clear focus on performance as a high priority goal. Not to take away from those who work on Python, but I think it's fair to say the effort has had orders of magnitude difference.
I don't have a deep enough understanding to say whether the nature of Python or Javascript makes one better suited for performance optimization than the other. Python is perhaps able to benefit from seeing what's been done with Javascript, although of course Javascript has stood on the shoulders of its own giants.
Good day. Is it the right time to talk to you about Common Lisp?
There are numerous active, moderately serious efforts to both optimize and/or JIT Python bytecode. I think AOT compilation is mostly out-of-scope for 100% compatibility, but again, there's lots of different efforts to compile either subset languages or subsets of programs.
"Kicking and screaming" suggests some reluctance to embrace this, but I think that's probably unfair: it's just hard.
That said I still use Python for CPU intensive tasks since in my experience Numpy/Scipy/Numba etc does a good job speeding up the CPU intensive parts of Python code.
That's not to say that Python will ever get there. My understanding is that the design of the language and leaky implementation details make generally compiling Python to fast machine code nearly impossible.
I dunno if Python is ever gonna be as fast as Java or C#, but we know it can be much better.
Python just scales terribly, no matter if you use multi-process or not. Java can get pretty good perf, but you'll need some libs or quite a bit of code to get nonblocking IO sending working well, or you're going to eat huge amounts of resources for moderate returns.
Node really excels at this use case. You can saturate the lines pretty easily.
Did I miss something? Does nodes/highland have good shared memory semantics these days?
I've always felt the best analogy to python concurrency was (node)js, but I admittedly haven't kept up all that well.
I vaguely recall Node used to run multiple threads under the hood for disk I/O, but it might use kqueue/epoll these days.
But I am curious to understand why you were not able to utilize the concurrency tools provided in Python.
A quick google search gave me these relevant resources
1. An intro to threading in Python (https://realpython.com/intro-to-python-threading/#conclusion...)
2. Speed Up Your Python Program With Concurrency (https://realpython.com/python-concurrency/)
3. Async IO in Python: A Complete Walkthrough (https://realpython.com/async-io-python/)
Forgive me for my naivety. This topic has been bothering me for quite a while.
Several people complain about the lack of threading in Python but I run into plenty of blogs and books on concurrency in Python.
Clearly there is a lack in my understanding of things.
In theory, multiprocessing could allow you to distribute the workload, but in a situation like OP describes -- just serving API requests based on a data structure -- the overhead of dispatching requests would likely be bigger than the cost of just handling the request in the first place. And your main server process is still a bottleneck for actually parsing the incoming requests and sending responses. So you're unlikely to see a significant benefit.
You can absolutely send stuff to a thread pool executor or process pool executor and then never await the returned value/never have it "return until interrupted, but the issues with shared memory (or really, the lack thereof in comparison to ex C) are still present to my understanding.
Then again, I mean you can always spin up a sqllite server or something on the same machine, but that's stupid heavy and more of a workaround than a solution. Super excited for nogil.
https://docs.python.org/3/library/concurrent.futures.html#co...
I mean it's not even right. Most parallel/concurrent pedants would consider multithreaded code to be "parallel" even if it is running on a single core.
I think the best thing is to talk about threads, because then you can distinguish e.g. OS threads and hardware threads.
Hm, I think running on a single core is the exact definition of what the "pedants" say is not parallel. If all you have is one core then you can't achieve parallelism under their definition.
I think the terminology is pretty well established now. But I do agree with you that it's a bad choice of words and that it's annoying, intelligent even, for people to pick a particular unintuitive definition and then go around brow-beating people for not using/understanding their definition.
If you're using these as technical terms which have specific technical definitions, then concurrency and parallelism are distinct but related concepts, and parallel computing means executing code simultaneously on separate execution units, not time sliced on a single core. So yes I am actually right about this. Parallel computing is defined this way in CS and it's not a matter of opinion.
The confusion here is parallelism vs concurrency. Parallelism is executing multiple tasks at once and concurrency is the composition of multiple tasks.
For example, imagine there is a woodshop with multiple people and there is only one hammer. The people would be working on their projects such as a chair, a table, etc. Everyone needs to use the hammer to continue their project.
If someone needed a hammer, they would take the single hammer and use it. There are still other projects going on but everyone else would have to wait until the hammer is free. This is concurrency but not parallelism.
If there are multiple hammers, then multiple people could use the hammer at the same time and their project continues. This is parallelism and concurrency.
The hammer here is the CPU and the multiple projects are threads. When you have Python concurrency, you are sharing the hammer across different projects, but it's still one hammer. This is useful for dealing with blocking I/O but not computing bottlenecks.
Let's say that one of the projects needs wood from another place. There is no point in this project to hold on to the hammer when waiting for wood. This is what those Python concurrency libraries are solving for. In real life, you have tasks waiting on other services such as getting customer info from a database. You don't want the task to be wasting the CPU cycles doing nothing, so we can pass the CPU to another task.
But this doesn't mean that we are using more of the CPU. We are still stuck with a single core. If we have a compute bottleneck such as calculating a lot of numbers, then the concurrency libraries don't help.
You might be wondering why Python only allows for a single hammer/CPU core. It's because it's very hard to get parallelism properly working, you can end up with your program stalling easily if you don't do it correctly. The underlying data structures of Python were never designed with that in mind because it was meant to be a scripting language where performance wasn't key. Python grew massive and people started to apply Python to areas where performance was key. It's amazing that Python got so far even with GIL IMO.
As an aside, you might read about "multiprocessing" Python where you can use multiple CPU cores. This is true but there are heavy overhead costs to this. This is like building brand-new workshops with single hammers to handle more projects. This post would get even longer if I explained what is a "process" but to put it shortly, it is how the OS, such as Windows or Linux, manages tasks. There is a lot of overhead with it because it is meant to work with all sorts of different programs written in different languages.
In the past, for read-only data, I’ve used a disk file and relied on the the OS page cache to keep it performant.
For read-write, using a raw file safely gets risky quickly. And alternative languages with parallelism runs rings around python.
So getting rid of the GIL and allowing parallelism will be a big boon.
You did not miss anything. The GIL prevents parallel multi threading.
Is it just down to corporate sponsorship?
It’s an impressive project.
If you're allowing writes to the shared data structure... I'd ask myself am I using the right tool for the job. A proper database server like postgres will handle concurrent writers much, much better than you could code up hastily. And it will handle failures, backups, storage, security, configuration, etc. far better than an ad hoc solution.
Quoting GP:
>> often CPU heavy
We have to take their word for it that it's actually CPU heavy work, but if they're not lying and not mistaken then asyncio would do nothing for them.