Why does ast.literal_eval('5 * 7') fail?
Why does the literal evaluation of 5 * 7
fail, while 5 + 7
doesn't?
import ast
print(ast.literal_eval('5 + 7'))
# -> 12
print(ast.literal_eval('5 * 7'))
# ->
Traceback (most recent call last):
...
ValueError: malformed node or string: <_ast.BinOp object at ...>
The documentation doesn't explain this.
I found that problem after answering this question on SO: Getting the result of a string.
ast.literal_eval()
accepts +
in the evaluated data because 5+2j
(complex number*) are valid literals. The same applies to -
. To keep the code simple, no attempt is made to exclude +
or -
as a binary operators.
No other operators are allowed; the function is supposed to only accept literals, not expressions.
In other words, that 5 + 7
works is a bug, but one that is hard to fix without breaking support for constructing complex numbers. The implementation limits the use to operands that are numbers, unary +
and -
, or other binary operators (so you can't use these to concatenate lists or produce a set difference).
Also see several related Python bugtracker entries: #25335 ast.literal_eval fails to parse numbers with leading "+", #22525 ast.literal_eval() doesn't do what the documentation says and #4907 ast.literal_eval does not properly handled complex numbers
* Technically speaking, 2j
is a valid literal; Python parses 5+2j
as int(5) binop(+) complex(0, 2)
, and only later produces a complex(5, 2)
object from the result, when actually executing the addition.
The question is not "why is *
not accepted" but rather "why is +
accepted at all".
ast.literal_eval
can parse literals, but not expressions. However, in Python, complex numbers are not expressed as a single literal value; instead they consist of the real part and imaginary part added together; the imaginary part is signalled with j
. literal_eval
thus needs to support binary +
and -
to support complex number constants such as 1 + 2j
or -3.4e-5 - 1.72e9j
.
In many versions, including Python 3.5, literal_eval
is much more lax than it needs to be - it accepts any chain of additions and subtractions for as long as both the left and right-hand sides evaluate to any number, thus (1 + 3) + 2 + (4 - 5)
is still parsed, even if it is not complex constant consisting of real +
imaginary part.
+
and -
are not accepted unconditionally: if you try to add 2 lists together, it will fail, even though it can parse list literals, and addition is defined for lists:
>>> ast.literal_eval('[1] + [2]')
Traceback (most recent call last):
...
ValueError: malformed node or string: <_ast.BinOp object at 0x7fdddbe785f8>
>>> ast.literal_eval('[1, 2]')
[1, 2]
>>> [1] + [2]
[1, 2]