Parallel Python: What is a callback?

In Parallel Python it has something in the submit function called a callback (documentation) however it doesn't seem to explain it too well. I've posted on their forum a couple days ago and I've not received a response. Would someone explain what a callback is and what it's used for?


A callback is a function provided by the consumer of an API that the API can then turn around and invoke (calling you back). If I setup a Dr.'s appointment, I can give them my phone number, so they can call me the day before to confirm the appointment. A callback is like that, except instead of just being a phone number, it can be arbitrary instructions like "send me an email at this address, and also call my secretary and have her put it in my calendar.

Callbacks are often used in situations where an action is asynchronous. If you need to call a function, and immediately continue working, you can't sit there wait for its return value to let you know what happened, so you provide a callback. When the function is done completely its asynchronous work it will then invoke your callback with some predetermined arguments (usually some you supply, and some about the status and result of the asynchronous action you requested).

If the Dr. is out of the office, or they are still working on the schedule, rather than having me wait on hold until he gets back, which could be several hours, we hang up, and once the appointment has been scheduled, they call me.

In this specific case, Parallel Python's submit function will invoke your callback with any arguments you supply and the result of func, once func has finished executing.


The relevant spot in the docs:

callback - callback function which will be called with argument 
        list equal to callbackargs+(result,) 
        as soon as calculation is done
callbackargs - additional arguments for callback function

So, if you want some code to be executed as soon as the result is ready, you put that code into a function and pass that function as the callback argument. If you don't need other arguments, it will be just, e.g.:

def itsdone(result):
  print "Done! result=%r" % (result,)
...
submit(..., callback=itsdone)

For more on the callback pattern in Python, see e.g. my presentation here.