Python functools lru_cache with instance methods: release object

How can I use functools.lru_cache inside classes without leaking memory?

In the following minimal example the foo instance won't be released although going out of scope and having no referrer (other than the lru_cache).

from functools import lru_cache
class BigClass:
    pass
class Foo:
    def __init__(self):
        self.big = BigClass()
    @lru_cache(maxsize=16)
    def cached_method(self, x):
        return x + 5

def fun():
    foo = Foo()
    print(foo.cached_method(10))
    print(foo.cached_method(10)) # use cache
    return 'something'

fun()

But foo and hence foo.big (a BigClass) are still alive

import gc; gc.collect()  # collect garbage
len([obj for obj in gc.get_objects() if isinstance(obj, Foo)]) # is 1

That means that Foo/BigClass instances are still residing in memory. Even deleting Foo (del Foo) will not release them.

Why is lru_cache holding on to the instance at all? Doesn't the cache use some hash and not the actual object?

What is the recommended way use lru_caches inside classes?

I know of two workarounds: Use per instance caches or make the cache ignore object (which might lead to wrong results, though)

Solution 1:

This is not the cleanest solution, but it's entirely transparent to the programmer:

import functools
import weakref

def memoized_method(*lru_args, **lru_kwargs):
    def decorator(func):
        @functools.wraps(func)
        def wrapped_func(self, *args, **kwargs):
            # We're storing the wrapped method inside the instance. If we had
            # a strong reference to self the instance would never die.
            self_weak = weakref.ref(self)
            @functools.wraps(func)
            @functools.lru_cache(*lru_args, **lru_kwargs)
            def cached_method(*args, **kwargs):
                return func(self_weak(), *args, **kwargs)
            setattr(self, func.__name__, cached_method)
            return cached_method(*args, **kwargs)
        return wrapped_func
    return decorator

It takes the exact same parameters as lru_cache, and works exactly the same. However it never passes self to lru_cache and instead uses a per-instance lru_cache.

Solution 2:

I will introduce methodtools for this use case.

pip install methodtools to install https://pypi.org/project/methodtools/

Then your code will work just by replacing functools to methodtools.

from methodtools import lru_cache
class Foo:
    @lru_cache(maxsize=16)
    def cached_method(self, x):
        return x + 5

Of course the gc test also returns 0 too.

Solution 3:

python 3.8 introduced the cached_property decorator in the functools module. when tested its seems to not retain the instances.

If you don't want to update to python 3.8 you can use the source code. All you need is to import RLock and create the _NOT_FOUND object. meaning:

from threading import RLock

_NOT_FOUND = object()

class cached_property:
    # https://github.com/python/cpython/blob/v3.8.0/Lib/functools.py#L930
    ...

Solution 4:

Simple wrapper solution

Here's a wrapper that will keep a weak reference to the instance:

import functools
import weakref

def weak_lru(maxsize=128, typed=False):
    'LRU Cache decorator that keeps a weak reference to "self"'
    def wrapper(func):

        @functools.lru_cache(maxsize, typed)
        def _func(_self, *args, **kwargs):
            return func(_self(), *args, **kwargs)

        @functools.wraps(func)
        def inner(self, *args, **kwargs):
            return _func(weakref.ref(self), *args, **kwargs)

        return inner

    return wrapper

Example

Use it like this:

class Weather:
    "Lookup weather information on a government website"

    def __init__(self, station_id):
        self.station_id = station_id

    @weak_lru(maxsize=10)
    def climate(self, category='average_temperature'):
        print('Simulating a slow method call!')
        return self.station_id + category

When to use it

Since the weakrefs add some overhead, you would only want to use this when the instances are large and the application can't wait for the older unused calls to age out of the cache.

Why this is better

Unlike the other answer, we only have one cache for the class and not one per instance. This is important if you want to get some benefit from the least recently used algorithm. With a single cache per method, you can set the maxsize so that the total memory use is bounded regardless of the number of instances that are alive.

Dealing with mutable attributes

If any of the attributes used in the method are mutable, be sure to add _eq_() and _hash_() methods:

class Weather:
    "Lookup weather information on a government website"

    def __init__(self, station_id):
        self.station_id = station_id

    def update_station(station_id):
        self.station_id = station_id

    def __eq__(self, other):
        return self.station_id == other.station_id

    def __hash__(self):
        return hash(self.station_id)