I'd like to point out that the Python standard library offers an abstraction over threads and processes that simplifies the kind of concurrent work described in the article: https://docs.python.org/dev/library/concurrent.futures.html
You can write the threaded example as:
import concurrent.futures
import itertools
import random
def generate_random(count):
return [random.random() for _ in range(count)]
if __name__ == "__main__":
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
executor.submit(generate_random, 10000000)
executor.submit(generate_random, 10000000)
# I guess we don't care about the results...
Changing this to use multiple processes instead of multiple threads is just a matter of s/ThreadPoolExecutor/ProcessPoolExecutor.You can also write this more idiomatically (and collect the combined results) as:
if __name__ == "__main__":
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
out_list = list(
executor.map(lambda _: random.random(), range(20000000)))
In this example case, this will be quite a bit slower because the work item (in this case generating a single random number) is trivial compared to the overhead of maintaining a work queue of 200000000 items - but in a more typical case where the work takes more than a millisecond then it is better to let the executor manage the division of labour.