I don't understand this python __del__ behaviour

You cannot assume that __del__ will ever be called - it is not a place to hope that resources are automagically deallocated. If you want to make sure that a (non-memory) resource is released, you should make a release() or similar method and then call that explicitly (or use it in a context manager as pointed out by Thanatos in comments below).

At the very least you should read the __del__ documentation very closely, and then you should probably not try to use __del__. (Also refer to the gc.garbage documentation for other bad things about __del__)

I'm providing my own answer because, while I appreciate the advice to avoid __del__, my question was how to get it to work properly for the code sample provided.

Short version: The following code uses weakref to avoid the circular reference. I thought I'd tried this before posting the question, but I guess I must have done something wrong.

import types, weakref

class Dummy():
    def __init__(self, name):
        self.name = name
    def __del__(self):
        print "delete",self.name

d2 = Dummy("d2")
def func(self):
    print "func called"
d2.func = types.MethodType(func, weakref.ref(d2)) #This works
#d2.func = func.__get__(weakref.ref(d2), Dummy) #This works too
d2.func()
del d2
d2 = None
print "after d2"

Longer version: When I posted the question, I did search for similar questions. I know you can use with instead, and that the prevailing sentiment is that __del__ is BAD.

Using with makes sense, but only in certain situations. Opening a file, reading it, and closing it is a good example where with is a perfectly good solution. You've gone a specific block of code where the object is needed, and you want to clean up the object and the end of the block.

A database connection seems to be used often as an example that doesn't work well using with, since you usually need to leave the section of code that creates the connection and have the connection closed in a more event-driven (rather than sequential) timeframe.

If with is not the right solution, I see two alternatives:

1. You make sure __del__ works (see this blog for a better description of weakref usage)
2. You use the atexit module to run a callback when your program closes. See this topic for example.

While I tried to provide simplified code, my real problem is more event-driven, so with is not an appropriate solution (with is fine for the simplified code). I also wanted to avoid atexit, as my program can be long-running, and I want to be able to perform the cleanup as soon as possible.

So, in this specific case, I find it to be the best solution to use weakref and prevent circular references that would prevent __del__ from working.

This may be an exception to the rule, but there are use-cases where using weakref and __del__ is the right implementation, IMHO.

Instead of del, you can use the with operator.

http://effbot.org/zone/python-with-statement.htm

just like with filetype objects, you could something like

with Dummy('d1') as d:
    #stuff
#d's __exit__ method is guaranteed to have been called

del doesn't call __del__

del in the way you are using removes a local variable. __del__ is called when the object is destroyed. Python as a language makes no guarantees as to when it will destroy an object.

CPython as the most common implementation of Python, uses reference counting. As a result del will often work as you expect. However it will not work in the case that you have a reference cycle.

d3 -> d3.func -> d3

Python doesn't detect this and so won't clean it up right away. And its not just reference cycles. If an exception is throw you probably want to still call your destructor. However, Python will typically hold onto to the local variables as part of its traceback.

The solution is not to depend on the __del__ method. Rather, use a context manager.

class Dummy:
   def __enter__(self):
       return self

   def __exit__(self, type, value, traceback):
       print "Destroying", self

with Dummy() as dummy:
    # Do whatever you want with dummy in here
# __exit__ will be called before you get here

This is guaranteed to work, and you can even check the parameters to see whether you are handling an exception and do something different in that case.

A full example of a context manager.

class Dummy(object):
    def __init__(self, name):
        self.name = name
    def __enter__(self):
        return self
    def __exit__(self, exct_type, exce_value, traceback):
        print 'cleanup:', d
    def __repr__(self):
        return 'Dummy(%r)' % (self.name,)

with Dummy("foo") as d:
    print 'using:', d

print 'later:', d