When and how to use Python's RLock

Reading through the Python docs I came across RLock.

Can someone explain to me (with example) a scenario in which RLock would be preferred to Lock?

With particular reference to:

• RLock's “recursion level”. How is this useful?
• A threads "ownership" of an RLock object
• Performance?

Solution 1:

This is one example where I see the use:

Useful when

1. you want to have thread-safe access from outside the class and use the same methods from inside the class:

   class X:
       def __init__(self):
           self.a = 1
           self.b = 2
           self.lock = threading.RLock()
   
       def changeA(self):
           with self.lock:
               self.a = self.a + 1
   
       def changeB(self):
           with self.lock:
               self.b = self.b + self.a
   
       def changeAandB(self):
           # you can use chanceA and changeB thread-safe!
           with self.lock:
               self.changeA() # a usual lock would block at here
               self.changeB()
   

2. for recursion more obvious:

   lock = threading.RLock()
   def a(...):
        with lock:
   
            a(...) # somewhere inside
   

   other threads have to wait until the first call of a finishes = thread ownership.

Performance

Usually, I start programming with the Lock and when case 1 or 2 occur, I switch to an RLock. Until Python 3.2 the RLock should be a bit slower because of the additional code. It uses Lock:

Lock = _allocate_lock # line 98 threading.py

def RLock(*args, **kwargs):
    return _RLock(*args, **kwargs)

class _RLock(_Verbose):

    def __init__(self, verbose=None):
        _Verbose.__init__(self, verbose)
        self.__block = _allocate_lock()

Thread Ownership

within the given thread you can acquire a RLock as often as you like. Other threads need to wait until this thread releases the resource again.

This is different to the Lock which implies 'function-call ownership'(I would call it this way): Another function call has to wait until the resource is released by the last blocking function even if it is in the same thread = even if it is called by the other function.

When to use Lock instead of RLock

When you make a call to the outside of the resource which you can not control.

The code below has two variables: a and b and the RLock shall be used to make sure a == b * 2

import threading
a = 0 
b = 0
lock = threading.RLock()
def changeAandB(): 
    # this function works with an RLock and Lock
    with lock:
        global a, b
        a += 1
        b += 2
        return a, b

def changeAandB2(callback):
    # this function can return wrong results with RLock and can block with Lock
    with lock:
        global a, b
        a += 1
        callback() # this callback gets a wrong value when calling changeAandB2
        b += 2
        return a, b

In changeAandB2 the Lock would be the right choice although it does block. Or one can enhance it with errors using RLock._is_owned(). Functions like changeAandB2 may occur when you have implemented an Observer pattern or a Publisher-Subscriber and add locking afterward.

Solution 2:

Here is another use case for RLock. Suppose you have a web-facing user interface that supports concurrent access, but you need to manage certain kinds of access to an external resource. For instance, you have to maintain consistency between objects in memory and objects in a database, and you have a manager class that controls access to the database, with methods that you must ensure get called in a specific order, and never concurrently.

What you can do is create an RLock and a guardian thread that controls access to the RLock by constantly acquiring it, and releasing only when signaled to. Then, you ensure all methods you need to control access to are made to obtain the lock before they run. Something like this:

def guardian_func():
    while True:
        WebFacingInterface.guardian_allow_access.clear()
        ResourceManager.resource_lock.acquire()
        WebFacingInterface.guardian_allow_access.wait()
        ResourceManager.resource_lock.release()

class WebFacingInterface(object):
    guardian_allow_access = Event()
    resource_guardian = Thread(None, guardian_func, 'Guardian', [])
    resource_manager = ResourceManager()

    @classmethod
    def resource_modifying_method(cls):
        cls.guardian_allow_access.set()
        cls.resource_manager.resource_lock.acquire()
        cls.resource_manager.update_this()
        cls.resource_manager.update_that()
        cls.resource_manager.resource_lock.release()

class ResourceManager(object):
    resource_lock = RLock()

    def update_this(self):
        if self.resource_lock.acquire(False):
            try:
                pass # do something
                return True

            finally:
                self.resource_lock.release()
        else:
            return False

    def update_that(self):
        if self.resource_lock.acquire(False):
            try:
                pass # do something else
                return True
            finally:
                self.resource_lock.release()
        else:
            return False

This way, you're ensured of the following things:

1. Once a thread acquires the resource lock, it can call the resource manager's protected methods freely, because RLock is recursive
2. Once the thread acquires the resource lock through the master method in the web facing interface, all access to protected methods in the manager will be blocked to other threads
3. Protected methods in the manager can only be accessed by first appealing to the guardian.