It looks like this downloads them one-by-one instead of downloading at the same time.
Is there a simple Python equivalent to Java's .parallelStream().forEach() that would allow these calls to easily be run in parallel?
Is there a simple Python equivalent to Java's .parallelStream().forEach() that would allow these calls to easily be run in parallel?
from multiprocessing import Pool
with Pool(8) as p:
p.map(function, sequence))I see that you explicitly specify 8 as the number of processes here. In Java, parallelStream() will pick a sane default for you if you haven't previously specified (based on the number of available processors, I believe). Is something like that possible in Python?
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor as executor:
executor.map(function, sequence)