ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

I just discovered a logical bug in my code which was causing all sorts of problems. I was inadvertently doing a bitwise AND instead of a logical AND.

I changed the code from:

r = mlab.csv2rec(datafile, delimiter=',', names=COL_HEADERS)
mask = ((r["dt"] >= startdate) & (r["dt"] <= enddate))
selected = r[mask]

TO:

r = mlab.csv2rec(datafile, delimiter=',', names=COL_HEADERS)
mask = ((r["dt"] >= startdate) and (r["dt"] <= enddate))
selected = r[mask]

To my surprise, I got the rather cryptic error message:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Why was a similar error not emitted when I use a bitwise operation - and how do I fix this?

Solution 1:

r is a numpy (rec)array. So r["dt"] >= startdate is also a (boolean) array. For numpy arrays the & operation returns the elementwise-and of the two boolean arrays.

The NumPy developers felt there was no one commonly understood way to evaluate an array in boolean context: it could mean True if any element is True, or it could mean True if all elements are True, or True if the array has non-zero length, just to name three possibilities.

Since different users might have different needs and different assumptions, the NumPy developers refused to guess and instead decided to raise a ValueError whenever one tries to evaluate an array in boolean context. Applying and to two numpy arrays causes the two arrays to be evaluated in boolean context (by calling __bool__ in Python3 or __nonzero__ in Python2).

Your original code

mask = ((r["dt"] >= startdate) & (r["dt"] <= enddate))
selected = r[mask]

looks correct. However, if you do want and, then instead of a and b use (a-b).any() or (a-b).all().

Solution 2:

I had the same problem (i.e. indexing with multi-conditions, here it's finding data in a certain date range). The (a-b).any() or (a-b).all() seem not working, at least for me.

Alternatively I found another solution which works perfectly for my desired functionality (The truth value of an array with more than one element is ambigous when trying to index an array).

Instead of using suggested code above, simply using a numpy.logical_and(a,b) would work. Here you may want to rewrite the code as

selected  = r[numpy.logical_and(r["dt"] >= startdate, r["dt"] <= enddate)]

Solution 3:

The reason for the exception is that and implicitly calls bool. First on the left operand and (if the left operand is True) then on the right operand. So x and y is equivalent to bool(x) and bool(y).

However the bool on a numpy.ndarray (if it contains more than one element) will throw the exception you have seen:

>>> import numpy as np
>>> arr = np.array([1, 2, 3])
>>> bool(arr)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The bool() call is implicit in and, but also in if, while, or, so any of the following examples will also fail:

>>> arr and arr
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

>>> if arr: pass
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

>>> while arr: pass
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

>>> arr or arr
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

There are more functions and statements in Python that hide bool calls, for example 2 < x < 10 is just another way of writing 2 < x and x < 10. And the and will call bool: bool(2 < x) and bool(x < 10).

The element-wise equivalent for and would be the np.logical_and function, similarly you could use np.logical_or as equivalent for or.

For boolean arrays - and comparisons like <, <=, ==, !=, >= and > on NumPy arrays return boolean NumPy arrays - you can also use the element-wise bitwise functions (and operators): np.bitwise_and (& operator)

>>> np.logical_and(arr > 1, arr < 3)
array([False,  True, False], dtype=bool)

>>> np.bitwise_and(arr > 1, arr < 3)
array([False,  True, False], dtype=bool)

>>> (arr > 1) & (arr < 3)
array([False,  True, False], dtype=bool)

and bitwise_or (| operator):

>>> np.logical_or(arr <= 1, arr >= 3)
array([ True, False,  True], dtype=bool)

>>> np.bitwise_or(arr <= 1, arr >= 3)
array([ True, False,  True], dtype=bool)

>>> (arr <= 1) | (arr >= 3)
array([ True, False,  True], dtype=bool)

A complete list of logical and binary functions can be found in the NumPy documentation:

• "Logic functions"
• "Binary operations"

Solution 4:

if you work with pandas what solved the issue for me was that i was trying to do calculations when I had NA values, the solution was to run:

df = df.dropna()

And after that the calculation that failed.

Solution 5:

Taking up @ZF007's answer, this is not answering your question as a whole, but can be the solution for the same error. I post it here since I have not found a direct solution as an answer to this error message elsewhere on Stack Overflow.

The error appears when you check whether an array was empty or not.

• if np.array([1,2]): print(1) --> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

• if np.array([1,2])[0]: print(1) --> no ValueError, but: if np.array([])[0]: print(1) --> IndexError: index 0 is out of bounds for axis 0 with size 0.

• if np.array([1]): print(1) --> no ValueError, but again will not help at an array with many elements.

• if np.array([]): print(1) --> DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use 'array.size > 0' to check that an array is not empty.

Doing so:

• if np.array([]).size: print(1) solved the error.