How to run asynchronous tasks in Python GObject Introspection apps

Your problem is a very common one, therefore there are tons of solutions (sheds, queues with multiprocessing or threading, worker pools, ...)

Since it is so common, there is also a python build-in solution (in 3.2, but backported here: http://pypi.python.org/pypi/futures) called concurrent.futures. 'Futures' are available in many languages, therefore python calls them the same. Here are the typical calls (and here is your full example, however, the db part is replaced by sleep, see below why).

from concurrent import futures
executor = futures.ProcessPoolExecutor(max_workers=1)
#executor = futures.ThreadPoolExecutor(max_workers=1)
future = executor.submit(slow_load)
future.add_done_callback(self.on_complete)

Now to your problem, which is much more complicated than your simple example suggests. In general you have threads or processes to solve this, but here is why your example is so complicated:

1. Most Python implementations have a GIL, which makes threads not fully utilize multicores. So: do not use threads with python!
2. The objects you want to return in slow_load from the DB are not pickelable, which means that they can not simply be passed between processes. So: no multiprocessing with softwarecenter results!
3. The library you call (softwarecenter.db) is not threadsafe (seems to include gtk or similar), therefore calling these methods in a thread results in strange behaviour (in my test, everything from 'it works' over 'core dump' to simple quitting without results). So: no threads with softwarecenter.
4. Every asynchronous callback in gtk should not do anything except sheduling a callback which will be called in the glib mainloop. So: no print, no gtk state changes, except adding a callback!
5. Gtk and alike does not work with threads out of the box. You need to do threads_init, and if you call a gtk or alike method, you have to protect that method (in earlier versions this was gtk.gdk.threads_enter(), gtk.gdk.threads_leave(). see for example gstreamer: http://pygstdocs.berlios.de/pygst-tutorial/playbin.html).

I can give you the following suggestion:

1. Rewrite your slow_load to return pickelable results and use futures with processes.
2. Switch from softwarecenter to python-apt or similar (you probably don't like that). But since your employed by Canonical, you could ask the softwarecenter developers directly to add documention to their software (e.g. stating that it is not thread safe) and even better, making softwarecenter threadsafe.

As a note: the solutions given by the others (Gio.io_scheduler_push_job, async_call) do work with time.sleep but not with softwarecenter.db. This is, because it all boils down to threads or processes and threads to not work with gtk and softwarecenter.

Here's another option using GIO's I/O Scheduler (I've never used it before from Python, but the example below seems to run fine).

from gi.repository import GLib, Gio, GObject
import time

def slow_stuff(job, cancellable, user_data):
    print "Slow!"
    for i in xrange(5):
        print "doing slow stuff..."
        time.sleep(0.5)
    print "finished doing slow stuff!"
    return False # job completed

def main():
    GObject.threads_init()
    print "Starting..."
    Gio.io_scheduler_push_job(slow_stuff, None, GLib.PRIORITY_DEFAULT, None)
    print "It's running async..."
    GLib.idle_add(ui_stuff)
    GLib.MainLoop().run()

def ui_stuff():
    print "This is the UI doing stuff..."
    time.sleep(1)
    return True

if __name__ == '__main__':
    main()

You can also use GLib.idle_add(callback) to call the long running task once the GLib Mainloop finishes all it's higher priority events (which I believe includes building the UI).

Use the introspected Gio API to read a file, with its asynchronous methods, and when making the initial call, do it as a timeout with GLib.timeout_add_seconds(3, call_the_gio_stuff) where call_the_gio_stuff is a function which returns False.

The timeout here is necessary to add (a different number of seconds may be required, though), because while the Gio async calls are asynchronous, they are not non-blocking, meaning that the heavy disk activity of reading a large file, or large number of files, can result in blocked UI, as the UI and I/O are still in the same (main) thread.

If you want to write your own functions to be async, and integrate with the main loop, using Python's file I/O APIs, you'll have to write the code as a GObject, or to pass callbacks around, or use python-defer to help you do it. But it's best to use Gio here, as it can bring you a lot of nice features, especially if you're doing file open/save stuff in the UX.