PyCharm type hinting custom list iteration

So I created a custom type that subclasses from list, which works well in most cases; though I noticed that Pycharm has trouble figuring out type hint when iterating over an object of the custom class.

Here's how I define it:

class MyList(list):

    def as_tuple(self):
        return tuple(self)

Then to test with, I just create a simple class A and then create a bunch of A objects:

from dataclasses import dataclass

@dataclass
class A:
    f1: str
    f2: int


objects: MyList[A] = MyList([
    A('hello', 111),
    A('world', 222)
])

I think the type hint might be unnecessary, as it looks like PyCharm correctly identifies it as MyList[A] in any case.

Accessing an element appears to work fine for type hinting purposes:

But PyCharm can't seem to offer type hints when iterating over the list type:

What I ended up doing so far is using typing.Generic in the class definition. I found out I also need to implement my own __iter__ method in the class, simply so I can type annotate the return type; otherwise, it doesn't seem to work without it.

Here's my new changes so far:

from typing import Generic, TypeVar, Iterator


T = TypeVar('T')


class MyList(list, Generic[T]):

    def __iter__(self) -> Iterator[T]:
        return super().__iter__()

    ...

Which apparently seems to do the trick:

So the question is, is there an easier way to get type hint for iteration in PyCharm over a list subclass without needing to set it up the way I have above?

The other main concern I have is a slight hit to performance. For example, I've timed the iterating using the following code as an example, and found that the second approach I've used is slower; this is not surprising though, since I've implemented a custom __iter__, which introduces an additional method call.

from timeit import timeit

print(timeit("for o in objects: ...", globals=globals()))

I'm tempted to use something complicated like a metaclass solution or else maybe just re-assigning the MyList.__iter__ method outside the class definition, but curious to see if anyone has any alternatives to the approach that I've used above.

Solution 1:

You were on the right path but it's a little confusing how python type hinting works.

All you need to do is this and you dont need to override __iter__.

class MyList(list[T]):
    ...

Why this works is cause you are telling PyCharm that you are creating a class that inherits the abilities of a generic list.

On the other hand inheriting list and Generic you tell PyCharm you want to inherit the abilities of list but not the abilities of a generic list. Instead its a class that supports generics and has list abilities.

As for performance, this will still hurt performance and that's ok. Python is not known for being performant and if your code really needs to be fast then you should either look into writing it in Cython or C and importing it into your script or migrating completely away from Python.