Lesson 8 of 10 Patterns

Distributed Locks with Redis (and Pitfalls)

Build a Redis lock with SET NX PX and a token-checked Lua release, see what happens when a lock expires, and learn when to use fencing tokens instead.

35-45 min advanced 0/10 lessons done
Course contents Lesson 8 of 10 · 0 done
On this page (8 sections)
  1. What you will learn
  2. The code
  3. The three essential parts
  4. The uncomfortable part: expiry vs slow work
  5. Redlock and replication
  6. Key takeaways
  7. Exercises
  8. Quiz

What you will learn

  • The correct way to acquire and release a lock on a single Redis instance.
  • Why the lock needs a unique token and a TTL.
  • The limits of Redis locks, and fencing tokens for correctness.

The code

"""A single-instance Redis lock: SET NX PX with a token, and a safe Lua release."""
import os
import time
import uuid

import redis

r = redis.Redis(port=int(os.getenv("REDIS_PORT", 6379)), decode_responses=True)
r.flushdb()

RELEASE = r.register_script("""
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0
""")


def acquire(name, ttl_ms):
    token = str(uuid.uuid4())  # unique per holder, so we only ever delete our own lock
    if r.set(f"lock:{name}", token, nx=True, px=ttl_ms):
        return token
    return None


def release(name, token):
    return RELEASE(keys=[f"lock:{name}"], args=[token]) == 1


t1 = acquire("invoice:42", ttl_ms=300)
t2 = acquire("invoice:42", ttl_ms=300)
print("worker 1 acquired:", t1 is not None, "| worker 2 acquired:", t2 is not None)

time.sleep(0.35)  # worker 1 is slow: its lock expires
t2 = acquire("invoice:42", ttl_ms=300)
print("after expiry, worker 2 acquired:", t2 is not None)

print("worker 1 release (stale token):", release("invoice:42", t1))
print("worker 2 release:", release("invoice:42", t2))
print("lock key exists:", r.exists("lock:invoice:42"))

Output:

worker 1 acquired: True | worker 2 acquired: False
after expiry, worker 2 acquired: True
worker 1 release (stale token): False
worker 2 release: True
lock key exists: 0

The three essential parts

  1. Atomic acquire with expiry: SET lock:name token NX PX 30000. NX makes only one client succeed; PX ensures a crashed holder doesn’t block everyone forever. Setting the value and expiry in one command matters: SETNX followed by EXPIRE can leave a lock without expiry if the client dies in between.
  2. A unique token per holder (a UUID), stored as the value.
  3. Release only if you still own it: a Lua script compares the token and deletes in one atomic step. A plain DEL could delete a lock someone else now holds.

The demo shows why step 3 matters: worker 1 was slow, its lock expired, worker 2 acquired it, and worker 1’s late release correctly did nothing.

The uncomfortable part: expiry vs slow work

What if worker 1 is still running when its lock expires? Now two workers run the critical section at once. Causes include GC pauses, slow I/O and network delays. Mitigations:

  • Choose a TTL well above the expected work time, and extend the lock periodically (a watchdog) while working. The extension must also check the token.
  • Make the protected operation idempotent where possible.
  • For correctness-critical work, use fencing tokens: each lock grant comes with an increasing number (for example from INCR), and the storage being protected rejects writes carrying a lower number than one it has already seen. A stale holder’s writes are then refused.

Redlock and replication

With a single Redis primary and async replication, a lock written to the primary can be lost if the primary fails before replicating, and a second client could acquire it on the promoted replica. Redlock acquires the lock on a majority of N independent Redis nodes to reduce this risk. It’s debated: it still relies on timing assumptions, which is why fencing tokens are recommended whenever two holders at once would corrupt data.

Practical guidance:

  • Efficiency locks (avoid doing the same work twice; occasional overlap is harmless): a single-instance Redis lock is fine.
  • Correctness locks (overlap would corrupt data or double-charge): use a database transaction or constraint, or a coordination service like ZooKeeper/etcd, plus fencing tokens.

The ticket booking lesson in the LLD course shows the in-process and database versions of the same problem.

Key takeaways

  • Acquire with SET key token NX PX ttl in one command.
  • Release with a Lua script that deletes only if the token matches.
  • Locks can expire while work continues; extend them, make work idempotent, or use fencing tokens.
  • Use Redis locks for efficiency; for strict correctness rely on database guarantees or fencing.

Exercises

  1. Add an extend(name, token, ttl_ms) Lua script and a background thread that extends the lock every ttl/3.
  2. Implement fencing: INCR lock:invoice:42:fence on acquire, and make a fake storage reject lower tokens.
  3. Write a test where two threads compete for the lock 1,000 times; count how often both believe they hold it (it should be zero with long TTLs).

Quiz

  1. Why not use SETNX and then EXPIRE?
  2. What goes wrong if release is a plain DEL?
  3. What is a fencing token?
Show answers
  1. If the client crashes between the two commands, the lock never expires and blocks everyone. SET ... NX PX does both atomically.
  2. A client whose lock already expired could delete a lock now held by someone else.
  3. An increasing number issued with each lock grant; the protected resource rejects operations carrying a token lower than one it has already accepted.

Finished this lesson?

Mark it complete to track your progress (saved in this browser).