multiprocessing.Pool is cool to do parallel jobs in Python. But some tutorials only take Pool.map for example, in which they used special cases of function accepting single argument.
There are four choices to mapping jobs to process. Here are the differences:
Multi-args Concurrence Blocking Ordered-results
map no yes yes yes
apply yes no yes no
map_async no yes no yes
apply_async yes yes no no
In Python 3, a new function starmap can accept multiple arguments.
Note that map and map_async are called for a list of jobs in one time, but apply and apply_async can only called for one job. However, apply_async execute a job in background therefore in parallel. See examples:
# map
results = pool.map(worker, [1, 2, 3])
# apply
for x, y in [[1, 1], [2, 2]]:
results.append(pool.apply(worker, (x, y)))
def collect_result(result):
results.append(result)
# map_async
pool.map_async(worker, jobs, callback=collect_result)
# apply_async
for x, y in [[1, 1], [2, 2]]:
pool.apply_async(worker, (x, y), callback=collect_result)
Reference