Distributed Locks

Introduction

Distributed locks coordinate cron jobs, inventory holds, and idempotent workers across pods — SET key unique-token NX EX ttl is the foundation. Without unique token and Lua unlock, process A can delete process B's lock after slow GC pause.

Redlock (multiple independent Redis nodes) targets higher fault tolerance but remains debated — use fencing tokens with downstream resources for correctness. Spring Integration RedisLockRegistry wraps common patterns.

Locks are leases — always EX; never rely on DEL alone at end.

Understanding the topic

Key concepts

• SET lock:resource token NX EX seconds — acquire.
• Token must be globally unique (UUID).
• Lua unlock — DEL only if GET == token.
• Extend lease with Lua if work may exceed TTL.
• Redlock — quorum across N masters.
• Fencing token — monotonic ID passed to storage.

sequenceDiagram
  Client->>Redis: SET lock token NX EX 30
  Client->>Client: critical section
  Client->>Redis: Lua delete if token match

Step-by-step explanation

1. Client generates token, attempts SET NX EX.
2. Success = lock held until TTL or unlock.
3. Work runs; before TTL extend or finish.
4. Unlock script compares token then DEL.
5. Expired lock allows another acquirer — design idempotent work.

Syntax reference

Common commands

• NX — only one holder.
• EX — auto-release on crash.
• Never GET then DEL in app code.

SET lock:report:daily $(uuidgen) NX EX 300
# work...
EVAL "if redis.call('GET',KEYS[1])==ARGV[1] then return redis.call('DEL',KEYS[1]) else return 0 end" 1 lock:report:daily <token>

Informative example

Scheduled job lock with Spring — one pod runs nightly report:

@Service
public class ReportLock {
    private final StringRedisTemplate redis;

    public ReportLock(StringRedisTemplate redis) {
        this.redis = redis;
    }

    public boolean tryRunDailyReport() {
        String token = UUID.randomUUID().toString();
        Boolean ok = redis.opsForValue().setIfAbsent("lock:daily-report", token, Duration.ofMinutes(10));
        if (!Boolean.TRUE.equals(ok)) return false;
        try {
            generateReport();
            return true;
        } finally {
            String script = "if redis.call('GET',KEYS[1])==ARGV[1] then return redis.call('DEL',KEYS[1]) else return 0 end";
            redis.execute((RedisCallback<Long>) c -> c.scriptingCommands().eval(
                script.getBytes(), ReturnType.INTEGER, 1,
                "lock:daily-report".getBytes(), token.getBytes()));
        }
    }
}

setIfAbsent = SET NX. Work must finish before TTL or extend lease. Use ShedLock library for battle-tested scheduling locks.

Real-world use

Real-world use cases

• Singleton cron across Kubernetes replicas.
• Inventory hold during checkout window.
• Migration script single-runner guard.
• Leader election lite for batch processor.
• Rate-limited external API single caller.

Best practices

• Unique token per acquisition.
• Lua compare-and-del unlock always.
• TTL > p99 job duration with margin.
• Fencing token to DB writes if correctness critical.
• Idempotent job body — lock may expire mid-run.
• Monitor lock contention metrics.

Common mistakes

• SET NX without EX — deadlock on crash.
• Unlock without token check.
• Same lock key all environments — staging steals prod lock.
• Redlock without understanding debate and fencing.

Advanced interview questions

Summary

SET NX EX + unique token = lock acquire.