With concurrent.futures
http://docs.python.org/3/library/concurrent.futures.html#con...it could look like:
from concurrent.futures import ThreadPoolExecutor as Pool
from socket import getaddrinfo
def lookup(domain):
try:
result = getaddrinfo(domain, 80)
exception Exception as e:
print("error %s -> %s" % (domain, e))
else:
print("done %s -> %s" % (domain, result))
nconcurrent = 20
with open('domains.txt') as file, Pool(nconcurrent) as pool:
for domain in (line.strip() for line in file):
pool.submit(lookup, domain)
To run multiple processes instead of threads, change the import to ProcessPoolExecutor.
To support multiprocessing.Pool (for Python 2 where concurrent.futures is not in stdlib), replace pool.submit() with pool.apply_async() and use contextlib.closing() around the Pool().