In Python, how do I determine if an object is iterable?

Is there a method like isiterable? The only solution I have found so far is to call

hasattr(myObj, '__iter__')

But I am not sure how fool-proof this is.

Solution 1:

1. Checking for __iter__ works on sequence types, but it would fail on e.g. strings in Python 2. I would like to know the right answer too, until then, here is one possibility (which would work on strings, too):

   from __future__ import print_function
   
   try:
       some_object_iterator = iter(some_object)
   except TypeError as te:
       print(some_object, 'is not iterable')
   

   The iter built-in checks for the __iter__ method or in the case of strings the __getitem__ method.

2. Another general pythonic approach is to assume an iterable, then fail gracefully if it does not work on the given object. The Python glossary:

   Pythonic programming style that determines an object's type by inspection of its method or attribute signature rather than by explicit relationship to some type object ("If it looks like a duck and quacks like a duck, it must be a duck.") By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). Instead, it typically employs the EAFP (Easier to Ask Forgiveness than Permission) style of programming.

   ...

   try:
      _ = (e for e in my_object)
   except TypeError:
      print my_object, 'is not iterable'
   

3. The collections module provides some abstract base classes, which allow to ask classes or instances if they provide particular functionality, for example:

   from collections.abc import Iterable
   
   if isinstance(e, Iterable):
       # e is iterable
   

   However, this does not check for classes that are iterable through __getitem__.

Solution 2:

Duck typing

try:
    iterator = iter(theElement)
except TypeError:
    # not iterable
else:
    # iterable

# for obj in iterator:
#     pass

Type checking

Use the Abstract Base Classes. They need at least Python 2.6 and work only for new-style classes.

from collections.abc import Iterable   # import directly from collections for Python < 3.3

if isinstance(theElement, Iterable):
    # iterable
else:
    # not iterable

However, iter() is a bit more reliable as described by the documentation:

Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an __iter__() method, but it does not detect classes that iterate with the __getitem__() method. The only reliable way to determine whether an object is iterable is to call iter(obj).

Solution 3:

I'd like to shed a little bit more light on the interplay of iter, __iter__ and __getitem__ and what happens behind the curtains. Armed with that knowledge, you will be able to understand why the best you can do is

try:
    iter(maybe_iterable)
    print('iteration will probably work')
except TypeError:
    print('not iterable')

I will list the facts first and then follow up with a quick reminder of what happens when you employ a for loop in python, followed by a discussion to illustrate the facts.

Facts

1. You can get an iterator from any object o by calling iter(o) if at least one of the following conditions holds true:
   
   a) o has an __iter__ method which returns an iterator object. An iterator is any object with an __iter__ and a __next__ (Python 2: next) method.
   
   b) o has a __getitem__ method.

2. Checking for an instance of Iterable or Sequence, or checking for the attribute __iter__ is not enough.

3. If an object o implements only __getitem__, but not __iter__, iter(o) will construct an iterator that tries to fetch items from o by integer index, starting at index 0. The iterator will catch any IndexError (but no other errors) that is raised and then raises StopIteration itself.

4. In the most general sense, there's no way to check whether the iterator returned by iter is sane other than to try it out.

5. If an object o implements __iter__, the iter function will make sure that the object returned by __iter__ is an iterator. There is no sanity check if an object only implements __getitem__.

6. __iter__ wins. If an object o implements both __iter__ and __getitem__, iter(o) will call __iter__.

7. If you want to make your own objects iterable, always implement the __iter__ method.

for loops

In order to follow along, you need an understanding of what happens when you employ a for loop in Python. Feel free to skip right to the next section if you already know.

When you use for item in o for some iterable object o, Python calls iter(o) and expects an iterator object as the return value. An iterator is any object which implements a __next__ (or next in Python 2) method and an __iter__ method.

By convention, the __iter__ method of an iterator should return the object itself (i.e. return self). Python then calls next on the iterator until StopIteration is raised. All of this happens implicitly, but the following demonstration makes it visible:

import random

class DemoIterable(object):
    def __iter__(self):
        print('__iter__ called')
        return DemoIterator()

class DemoIterator(object):
    def __iter__(self):
        return self

    def __next__(self):
        print('__next__ called')
        r = random.randint(1, 10)
        if r == 5:
            print('raising StopIteration')
            raise StopIteration
        return r

Iteration over a DemoIterable:

>>> di = DemoIterable()
>>> for x in di:
...     print(x)
...
__iter__ called
__next__ called
9
__next__ called
8
__next__ called
10
__next__ called
3
__next__ called
10
__next__ called
raising StopIteration

Discussion and illustrations

On point 1 and 2: getting an iterator and unreliable checks

Consider the following class:

class BasicIterable(object):
    def __getitem__(self, item):
        if item == 3:
            raise IndexError
        return item

Calling iter with an instance of BasicIterable will return an iterator without any problems because BasicIterable implements __getitem__.

>>> b = BasicIterable()
>>> iter(b)
<iterator object at 0x7f1ab216e320>

However, it is important to note that b does not have the __iter__ attribute and is not considered an instance of Iterable or Sequence:

>>> from collections import Iterable, Sequence
>>> hasattr(b, '__iter__')
False
>>> isinstance(b, Iterable)
False
>>> isinstance(b, Sequence)
False

This is why Fluent Python by Luciano Ramalho recommends calling iter and handling the potential TypeError as the most accurate way to check whether an object is iterable. Quoting directly from the book:

As of Python 3.4, the most accurate way to check whether an object x is iterable is to call iter(x) and handle a TypeError exception if it isn’t. This is more accurate than using isinstance(x, abc.Iterable) , because iter(x) also considers the legacy __getitem__ method, while the Iterable ABC does not.

On point 3: Iterating over objects which only provide __getitem__, but not __iter__

Iterating over an instance of BasicIterable works as expected: Python constructs an iterator that tries to fetch items by index, starting at zero, until an IndexError is raised. The demo object's __getitem__ method simply returns the item which was supplied as the argument to __getitem__(self, item) by the iterator returned by iter.

>>> b = BasicIterable()
>>> it = iter(b)
>>> next(it)
0
>>> next(it)
1
>>> next(it)
2
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Note that the iterator raises StopIteration when it cannot return the next item and that the IndexError which is raised for item == 3 is handled internally. This is why looping over a BasicIterable with a for loop works as expected:

>>> for x in b:
...     print(x)
...
0
1
2

Here's another example in order to drive home the concept of how the iterator returned by iter tries to access items by index. WrappedDict does not inherit from dict, which means instances won't have an __iter__ method.

class WrappedDict(object): # note: no inheritance from dict!
    def __init__(self, dic):
        self._dict = dic

    def __getitem__(self, item):
        try:
            return self._dict[item] # delegate to dict.__getitem__
        except KeyError:
            raise IndexError

Note that calls to __getitem__ are delegated to dict.__getitem__ for which the square bracket notation is simply a shorthand.

>>> w = WrappedDict({-1: 'not printed',
...                   0: 'hi', 1: 'StackOverflow', 2: '!',
...                   4: 'not printed', 
...                   'x': 'not printed'})
>>> for x in w:
...     print(x)
... 
hi
StackOverflow
!

On point 4 and 5: iter checks for an iterator when it calls __iter__:

When iter(o) is called for an object o, iter will make sure that the return value of __iter__, if the method is present, is an iterator. This means that the returned object must implement __next__ (or next in Python 2) and __iter__. iter cannot perform any sanity checks for objects which only provide __getitem__, because it has no way to check whether the items of the object are accessible by integer index.

class FailIterIterable(object):
    def __iter__(self):
        return object() # not an iterator

class FailGetitemIterable(object):
    def __getitem__(self, item):
        raise Exception

Note that constructing an iterator from FailIterIterable instances fails immediately, while constructing an iterator from FailGetItemIterable succeeds, but will throw an Exception on the first call to __next__.

>>> fii = FailIterIterable()
>>> iter(fii)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: iter() returned non-iterator of type 'object'
>>>
>>> fgi = FailGetitemIterable()
>>> it = iter(fgi)
>>> next(it)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/iterdemo.py", line 42, in __getitem__
    raise Exception
Exception

On point 6: __iter__ wins

This one is straightforward. If an object implements __iter__ and __getitem__, iter will call __iter__. Consider the following class

class IterWinsDemo(object):
    def __iter__(self):
        return iter(['__iter__', 'wins'])

    def __getitem__(self, item):
        return ['__getitem__', 'wins'][item]

and the output when looping over an instance:

>>> iwd = IterWinsDemo()
>>> for x in iwd:
...     print(x)
...
__iter__
wins

On point 7: your iterable classes should implement __iter__

You might ask yourself why most builtin sequences like list implement an __iter__ method when __getitem__ would be sufficient.

class WrappedList(object): # note: no inheritance from list!
    def __init__(self, lst):
        self._list = lst

    def __getitem__(self, item):
        return self._list[item]

After all, iteration over instances of the class above, which delegates calls to __getitem__ to list.__getitem__ (using the square bracket notation), will work fine:

>>> wl = WrappedList(['A', 'B', 'C'])
>>> for x in wl:
...     print(x)
... 
A
B
C

The reasons your custom iterables should implement __iter__ are as follows:

1. If you implement __iter__, instances will be considered iterables, and isinstance(o, collections.abc.Iterable) will return True.
2. If the object returned by __iter__ is not an iterator, iter will fail immediately and raise a TypeError.
3. The special handling of __getitem__ exists for backwards compatibility reasons. Quoting again from Fluent Python:

That is why any Python sequence is iterable: they all implement __getitem__ . In fact, the standard sequences also implement __iter__, and yours should too, because the special handling of __getitem__ exists for backward compatibility reasons and may be gone in the future (although it is not deprecated as I write this).

Solution 4:

I've been studying this problem quite a bit lately. Based on that my conclusion is that nowadays this is the best approach:

from collections.abc import Iterable   # drop `.abc` with Python 2.7 or lower

def iterable(obj):
    return isinstance(obj, Iterable)

The above has been recommended already earlier, but the general consensus has been that using iter() would be better:

def iterable(obj):
    try:
        iter(obj)
    except Exception:
        return False
    else:
        return True

We've used iter() in our code as well for this purpose, but I've lately started to get more and more annoyed by objects which only have __getitem__ being considered iterable. There are valid reasons to have __getitem__ in a non-iterable object and with them the above code doesn't work well. As a real life example we can use Faker. The above code reports it being iterable but actually trying to iterate it causes an AttributeError (tested with Faker 4.0.2):

>>> from faker import Faker
>>> fake = Faker()
>>> iter(fake)    # No exception, must be iterable
<iterator object at 0x7f1c71db58d0>
>>> list(fake)    # Ooops
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/.../site-packages/faker/proxy.py", line 59, in __getitem__
    return self._factory_map[locale.replace('-', '_')]
AttributeError: 'int' object has no attribute 'replace'

If we'd use insinstance(), we wouldn't accidentally consider Faker instances (or any other objects having only __getitem__) to be iterable:

>>> from collections.abc import Iterable
>>> from faker import Faker
>>> isinstance(Faker(), Iterable)
False

Earlier answers commented that using iter() is safer as the old way to implement iteration in Python was based on __getitem__ and the isinstance() approach wouldn't detect that. This may have been true with old Python versions, but based on my pretty exhaustive testing isinstance() works great nowadays. The only case where isinstance() didn't work but iter() did was with UserDict when using Python 2. If that's relevant, it's possible to use isinstance(item, (Iterable, UserDict)) to get that covered.

Solution 5:

Since Python 3.5 you can use the typing module from the standard library for type related things:

from typing import Iterable

...

if isinstance(my_item, Iterable):
    print(True)