Scope of lambda functions and their parameters?

Solution 1:

When a lambda is created, it doesn't make a copy of the variables in the enclosing scope that it uses. It maintains a reference to the environment so that it can look up the value of the variable later. There is just one m. It gets assigned to every time through the loop. After the loop, the variable m has value 'mi'. So when you actually run the function you created later, it will look up the value of m in the environment that created it, which will by then have value 'mi'.

One common and idiomatic solution to this problem is to capture the value of m at the time that the lambda is created by using it as the default argument of an optional parameter. You usually use a parameter of the same name so you don't have to change the body of the code:

for m in ('do', 're', 'mi'):
    funcList.append(lambda m=m: callback(m))

Solution 2:

The problem here is the m variable (a reference) being taken from the surrounding scope. Only parameters are held in the lambda scope.

To solve this you have to create another scope for lambda:

def callback(msg):
    print msg

def callback_factory(m):
    return lambda: callback(m)

funcList=[]
for m in ('do', 're', 'mi'):
    funcList.append(callback_factory(m))
for f in funcList:
    f()

In the example above, lambda also uses the surounding scope to find m, but this time it's callback_factory scope which is created once per every callback_factory call.

Or with functools.partial:

from functools import partial

def callback(msg):
    print msg

funcList=[partial(callback, m) for m in ('do', 're', 'mi')]
for f in funcList:
    f()

Solution 3:

Python does uses references of course, but it does not matter in this context.

When you define a lambda (or a function, since this is the exact same behavior), it does not evaluate the lambda expression before runtime:

# defining that function is perfectly fine
def broken():
    print undefined_var

broken() # but calling it will raise a NameError

Even more surprising than your lambda example:

i = 'bar'
def foo():
    print i

foo() # bar

i = 'banana'

foo() # you would expect 'bar' here? well it prints 'banana'

In short, think dynamic: nothing is evaluated before interpretation, that's why your code uses the latest value of m.

When it looks for m in the lambda execution, m is taken from the topmost scope, which means that, as others pointed out; you can circumvent that problem by adding another scope:

def factory(x):
    return lambda: callback(x)

for m in ('do', 're', 'mi'):
    funcList.append(factory(m))

Here, when the lambda is called, it looks in the lambda' definition scope for a x. This x is a local variable defined in factory's body. Because of this, the value used on lambda execution will be the value that was passed as a parameter during the call to factory. And doremi!

As a note, I could have defined factory as factory(m) [replace x by m], the behavior is the same. I used a different name for clarity :)

You might find that Andrej Bauer got similar lambda problems. What's interesting on that blog is the comments, where you'll learn more about python closure :)

Solution 4:

Yes, that's a problem of scope, it binds to the outer m, whether you are using a lambda or a local function. Instead, use a functor:

class Func1(object):
    def __init__(self, callback, message):
        self.callback = callback
        self.message = message
    def __call__(self):
        return self.callback(self.message)
funcList.append(Func1(callback, m))

Solution 5:

the soluiton to lambda is more lambda

In [0]: funcs = [(lambda j: (lambda: j))(i) for i in ('do', 're', 'mi')]

In [1]: funcs
Out[1]: 
[<function __main__.<lambda>>,
 <function __main__.<lambda>>,
 <function __main__.<lambda>>]

In [2]: [f() for f in funcs]
Out[2]: ['do', 're', 'mi']

the outer lambda is used to bind the current value of i to j at the

each time the outer lambda is called it makes an instance of the inner lambda with j bound to the current value of i as i's value