Python Multiprocessing: What's the difference between map and imap?

That is the difference. One reason why you might use imap instead of map is if you wanted to start processing the first few results without waiting for the rest to be calculated. map waits for every result before returning.

As for chunksize, it is sometimes more efficient to dole out work in larger quantities because every time the worker requests more work, there is IPC and synchronization overhead.

imap is from itertools module which is used for fast and memory efficiency in python.Map will return the list where as imap returns the object which generates the values for each iterations(In python 2.7).The below code blocks will clear the difference.

Map returns the list can be printed directly

 from itertools import *
    from math import *

    integers = [1,2,3,4,5]
    sqr_ints = map(sqrt, integers)
    print (sqr_ints)

imap returns object which is converted to list and printed.

from itertools import *
from math import *

integers = [1,2,3,4,5]
sqr_ints = imap(sqrt, integers)
print list(sqr_ints)

Chunksize will make the iterable to be split into pieces of specified size(approximate) and each piece is submitted as a separate task.

With imap, forked calls are done in parallel, not one after another sequentially. For example, below you're hitting say three exchanges to get order books. Instead of hitting exchange 1, then exchange 2, then exchange 3 sequentially, imap.pool calls are non-blocking and goes straight to all three exchanges to fetch order books as soon as you call.

from pathos.multiprocessing import ProcessingPool as Pool
pool = Pool().imap
self.pool(self.getOrderBook, Exchanges, Tickers)