They’re actually not. They are native threads with high lock contention.
Async is arguably fibers, as are greenthreads in libraries like gevent or eventlet.
> But if you want to use real threads, multiprocessing's "processes" are actually system threads.
They’re system threads running in separate memory spaces. Also known as… processes.
import numba as nb
from concurrent.futures import ThreadPoolExecutor
from multiprocessing import cpu_count
@nb.jit(nogil=True)
def slow_calculation(x):
out = 0
for i in range(x):
out += i**0.01
return out
ex = ThreadPoolExecutor(max_workers=cpu_count())
futures = [ex.submit(slow_calculation, 100_000_000_000+i) for i in range(cpu_count())]Even without requiring the GIL, these are still child threads of the main process, correct? And because of that, wouldn't the OS keep them all on the same core? And if that's the case, would ProcessPoolExecutor solve that problem?