List comprehension scope error from Python debugger

Solution 1:

In Python 3, you have to use the interact command in pdb before you can access any non-global variables due to a change in the way comprehensions are implemented.

>>> def foo(): [][0]
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in foo
IndexError: list index out of range
>>> import pdb;pdb.pm()
> <stdin>(1)foo()
(Pdb) x = 4
(Pdb) [x for _ in range(2)]
*** NameError: name 'x' is not defined
(Pdb) interact
*interactive*
>>> [x for _ in range(2)]
[4, 4]
>>> 

Solution 2:

pdb seems to be running the code with:

eval(compiled_code, globals(), locals())

(or maybe even just eval(string, globals(), locals())).

Unfortunately, on compilation Python doesn't know of the local variables. This doesn't matter normally:

import dis

dis.dis(compile("x", "", "eval"))
#>>>   1           0 LOAD_NAME                0 (x)
#>>>               3 RETURN_VALUE

but when another scope is introduced, such as with a list comprehension of lambda, this compiles badly:

dis.dis(compile("(lambda: x)()", "", "eval"))
#>>>   1           0 LOAD_CONST               0 (<code object <lambda> at 0x7fac20708d20, file "", line 1>)
#>>>               3 LOAD_CONST               1 ('<lambda>')
#>>>               6 MAKE_FUNCTION            0
#>>>               9 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
#>>>              12 RETURN_VALUE

# The code of the internal lambda
dis.dis(compile("(lambda: x)()", "", "eval").co_consts[0])
#>>>   1           0 LOAD_GLOBAL              0 (x)
#>>>               3 RETURN_VALUE

Note how that's a LOAD_GLOBAL where x is in the local scope.

Here's a totally stupid hack to get around it:

(Pdb) eval("(lambda: x)()", vars())
[1, 2, 3, 3, 4]