Theoretic Mutex API and Use

In the last post I outlined the need for Mutexes for Apex. But what might this look like if implemented? In this post I’ll set out a possible API and then show how it could be used.

Before we go any further I should note this is a thought exercise. I have no knowledge of any Mutex API coming. I have no reason to believe this specific API is going to be implemented. If Salesforce happen to add Mutexes to Apex and it happens to look anything like this then I just got lucky!

System.Mutex {
// Creates an instance of Mutex. Every call to the constructor with the same name returns the same object
Mutex(String name);
// Returns true if the named Mutex is already locked
boolean isLocked();
// Attempts to lock the Mutex. If the Mutex is unlocked locks and returns true. If the Mutex is locked waits at most timeout milliseconds for it to unlock. Throws an Exception if the call has to wait longer than timeout milliseconds.
// If locked and has been locked for a long time (platform defined) without extendLock will allow locking even though locked (to allow for limits exceptions causing unlock to fail)
void lock(Long timeout);
// Extends a lock preventing timeout allowing locking of a locked lock
void extendLock();
// Release a lock
void unlock()
}

There are a few interesting things to note about this API. Mutexes are named. Only one Mutex with the same name can exist.

Attempting to lock a lock has some interesting behaviour. A timeout is passed to the lock method. If the lock cannot be locked in this time period then an exception is thrown. This may be unnecessary - the platform could enforce this instead. In fact it might have to depending on what conditions are placed around the lock method. If DML is allowed before the lock method is called then the timeout would have to be very short or possibly zero. Most likely DML would not be allowed before calling lock and then a timeout can be allowed without causing transaction locking issues.

However due to the nature of the platform there would still be concerns around locks getting locked and never unlocked. If a transaction locks a lock, does some DML then unlocks the lock all is good - a sensitive resource is guarded from concurrent access by a lock. But what if the transaction hits a governor limit and gets terminated? The lock would remain locked. For normal DML we’d expect that the transaction rolls back (or actually was never committed to the database). Locks would have to operate differently. They would have to actually lock in a persistent fashion whenever lock is called.

To protect against locks staying locked some timeout on the length of locks is also required. The system can manage this behind the scenes. However long running processes in chains of Queueables would have to be able to keep a lock locked. Hence the extendLock method.

The isLocked method would have to obey these rules too. Or a canLock method would have to be provided which could result in isLocked and canLock both returning true. For this we’ll assume isLocked returns false if the lock times out without being extended or unlocked.

Now that we have a minimal API that takes the platform foibles into account what could we do with it? Let’s imagine a not too uncommon scenario. We have some sort of action that we want to perform in a single threaded manner. The exact operation is not important. Even if multiple users at the same time are enqueuing actions the actions have to be performed from a single threaded queue in a FIFO manner.

We want to process in the background. We want to use as modern Apex as we can so we’ll use Queueables. When an action is enqueued if there is no Queueable running we should write to the queue (an sObject) and start a Queueable. If there is a Queueable running we write to the queue and don’t start a Queueable. When a Queueable ends if there are unprocessed items it chains another Queueable. If there are no more items the chain ends.

There are multiple possible race conditions here. Multiple enqueues could happen at the same time. Only one should start the Queueable. Enqueue could happen just as a Queueable is ending. At this point the enqueue could think a Queueable is running and the Queueable might think there are no actions waiting. Resulting in items in the queue but no process. A Mutex can solve all this!

global class FIFOQueue {
  // Enqueue a list of QueueItems__c (not defined here for brevity) and process FIFO
  global void enqueue(String queueName, List<QueueItems__c> items) {
    // All queue access must be protected
    Mutex qMut = new Mutex(queueName+’queue’); // Intentionally let this throw if it times out
    qMut.lock(TIMEOUT);
    Mutex pMut = new Mutex(queueName+’_process’);
    boolean startQueueable = false;
    if (!pMut.isLocked) {
      // This is the only place this Mutex gets locked and is protected by the above Mutex. Only 1 thread can be here so only 1 process can ever get started
      pMut.lock(TIMEOUT);
      startQueueable = true;
    }
    // All Mutex locking done. Can do DML now
    insert items;
    if (startQueueable) {
      System.enqueueJob(new FIFOQueueRunner(), 0);
    }
  }

  // Queueable inner class to execute the item
  private class FIFOQueueRunner implements Queueable {
    public void execute(QueueableContext context) {
      Mutex qMut = new Mutex(queueName+’queue’); // Intentionally let this throw if it times out
      Mutex pMut = new Mutex(queueName+’_process’);
      pMut.extendLock();
      qMut.lock(TIMEOUT);
      // Read the items from the queue to process
      qMut.unlock();
      // Process the items however is necessary
      qMut.lock(TIMEOUT);
      // Update the processed items as necessary and read if there are more items to process
      If (moreItemsToProcess) {
              System.enqueueJob(new FIFOQueueRunner(), 0);
      } else {
         pMut.unlock();
      }
      qMut.unlock();
    }
  }
}

This code is far from perfect. In the real world we’d probably use a Finalizer to do the update of the queueable items allowing for us to mark them as errors if necessary and enqueue again from the finalizer. However hopefully this gives an idea of how Mutexes could be used on the platform.

It is possible to build Mutexes out of sObjects, forUpdate SOQL and a lot of care (and no doubt quite a bit of luck at runtime). But they always feel like the luck might run out. Platform Mutexes would remove this fear. If you think Mutexes are a good idea vote for this Idea on Ideas Exchange.