How to reload a module's function in Python?
As of today, the proper way of doing this is:
import sys, importlib
importlib.reload(sys.modules['foo'])
from foo import bar
Tested on python 2.7, 3.5, 3.6.
What you want is possible, but requires reloading two things... first reload(foo)
, but then you also have to reload(baz)
(assuming baz
is the name of the module containing the from foo import bar
statement).
As to why... When foo
is first loaded, a foo
object is created, containing a bar
object. When you import bar
into the baz
module, it stores a reference to bar
. When reload(foo)
is called, the foo
object is blanked, and the module re-executed. This means all foo
references are still valid, but a new bar
object has been created... so all references that have been imported somewhere are still references to the old bar
object. By reloading baz
, you cause it to reimport the new bar
.
Alternately, you can just do import foo
in your module, and always call foo.bar()
. That way whenever you reload(foo)
, you'll get the newest bar
reference.
NOTE: As of Python 3, the reload function needs to be imported first, via from importlib import reload
Hot reloading is not something you can do in Python reliably without blowing up your head. You literally cannot support reloading without writing code special ways, and trying to write and maintain code that supports reloading with any sanity requires extreme discipline and is too confusing to be worth the effort. Testing such code is no easy task either.
The solution is to completely restart the Python process when code has changed. It is possible to do this seamlessly, but how depends on your specific problem domain.
While reloading of functions is not a feature of the reload
function, it is still possible. I would not recommend doing it in production, but here is how it works:
The function you want to replace is a object somewhere in memory, and your code might hold many references to it (and not to the name of the function). But what that function actually does when called is not saved in that object, but in another object that, in turn, is referenced by the function object, in its attribute __code__
. So as long as you have a reference to the function, you can update its code:
Module mymod:
from __future__ import print_function
def foo():
print("foo")
Other module / python session:
>>> import mymod
>>> mymod.foo()
foo
>>> old_foo_ref = mymod.foo
>>> # edit mymod.py to make function "foo" print "bar"
>>> reload(mymod)
<module 'mymod' from 'mymod.py'>
>>> old_foo_ref() # old reference is running old code
foo
>>> mymod.foo() # reloaded module has new code
bar
>>> old_foo_ref.__code__ = mymod.foo.__code__
>>> old_foo_ref() # now the old function object is also running the new code
bar
>>>
Now, in case you do not have a reference to the old function (i.e. a lambda
passed into another function), you can try to get a hold of it with the gc
module, by searching the list of all objects:
>>> def _apply(f, x):
... return lambda: f(x)
...
>>> tmp = _apply(lambda x: x+1, 1) # now there is a lambda we don't have a reference to
>>> tmp()
2
>>> import gc
>>> lambdas = [obj for obj in gc.get_objects() if getattr(obj,'__name__','') == '<lambda>'] # get all lambdas
[<function <lambda> at 0x7f315bf02f50>, <function <lambda> at 0x7f315bf12050>]
# i guess the first one is the one i passed in, and the second one is the one returned by _apply
# lets make sure:
>>> lambdas[0].__code__.co_argcount
1 # so this is the "lambda x: ..."
>>> lambdas[1].__code__.co_argcount
0 # and this is the "lambda: ..."
>>> lambdas[0].__code__ = (lambda x: x+2).__code__ # change lambda to return arg + 2
>>> tmp()
3 #
>>>