multiprocessing.Pool is cool to do parallel jobs in Python. But some tutorials only take
Pool.map for example, in which they used special cases of function accepting single argument.
There are four choices to mapping jobs to process. Here are the differences:
Multi-args Concurrence Blocking Ordered-results map no yes yes yes apply yes no yes no map_async no yes no yes apply_async yes yes no no
In Python 3, a new function
starmap can accept multiple arguments.
map_async are called for a list of jobs in one time, but
apply_async can only called for one job. However,
apply_async execute a job in background therefore in parallel. See examples:
# map results = pool.map(worker, [1, 2, 3]) # apply for x, y in [[1, 1], [2, 2]]: results.append(pool.apply(worker, (x, y))) def collect_result(result): results.append(result) # map_async pool.map_async(worker, jobs, callback=collect_result) # apply_async for x, y in [[1, 1], [2, 2]]: pool.apply_async(worker, (x, y), callback=collect_result)