Java Memory Model

Introduction

Two payment-processing threads can both "see" a stale balance because the JVM allows CPU caches and compiler reordering unless you establish a happens-before relationship. The Java Memory Model (JMM) defines which writes are visible to which reads across threads — the contract behind synchronized, volatile, and java.util.concurrent.

This lesson maps memory regions — heap, stack, Metaspace — to banking domain objects, then dives into visibility, happens-before, and volatile semantics. Without JMM literacy, you cannot explain why a flag check loop never terminates, why double-checked locking failed pre-Java 5, or why a payment status field updated in one thread is invisible to a reconciliation worker.

JMM is prerequisite for correct concurrency in ledger services — races without exceptions are JMM bugs.

Business problem

Visibility bugs corrupt financial state without stack traces:

• Stale payment flag: Worker sets processed=true; reconciliation thread never sees it — duplicate settlement attempt.
• Partially constructed Account: Thread publishes reference before constructor finishes — reader sees zero balance on live account.
• Non-volatile shutdown flag: Batch job loop never stops on deploy — JVM optimizes away re-read of plain boolean.
• Metaspace OOM: Dynamic proxy classes from Spring AOP exhaust class metadata — all payment pods crash simultaneously.

Why this topic exists

Hardware and compilers optimize aggressively — without a formal memory model, multithreaded Java would have no portable correctness guarantees.

• CPU caches: Each core has local cache — writes may not reach main memory immediately without synchronization.
• Compiler reordering: JIT reorders instructions for performance — legal unless JMM forbids visible reordering.
• Portable semantics: JMM defines happens-before — same behavior on ARM, x86, and cloud VMs.
• API contract: synchronized, volatile, final, Thread.start, concurrent utilities all establish happens-before edges.

Core concepts

Six JMM pillars for enterprise engineers:

• Heap: Shared among threads — Payment objects, account balances, concurrent collections live here.
• Stack: Per-thread — local variables, method frames; not shared; references point to heap objects.
• Metaspace: Class metadata (Java 8+ replaced PermGen) — Spring proxies, generated classes; unbounded growth causes OOM.
• Visibility: When thread B sees thread A's write — requires happens-before, not accidental timing.
• Happens-before: Ordering guarantee — if A happens-before B, B sees A's writes.
• volatile: Visibility + ordering for reads/writes — not atomicity for compound operations like increment.

Internal architecture

Memory layout for a PaymentService instance:

Per JVM process:
┌─────────────────────────────────────────┐
│ Metaspace (class metadata)              │
│   PaymentService.class, Spring proxies  │
├─────────────────────────────────────────┤
│ Heap (shared — all threads)             │
│   Payment objects, ConcurrentHashMap    │
│   Account balances (mutable — needs sync)│
├─────────────────────────────────────────┤
│ Thread 1 Stack    │ Thread 2 Stack      │
│   local amount    │   local txnId       │
│   ref ────────────┼──▶ same Payment    │
└─────────────────────────────────────────┘

Happens-before edges (examples):
  unlock(monitor) ──hb──▶ lock(same monitor)
  volatile write ──hb──▶ volatile read (same var)
  Thread.start() ──hb──▶ run() body actions
  final field write in ctor ──hb──▶ final read after safe publish

Visibility failure (BAD):
  Thread-1: paymentProcessed = true;  // plain field
  Thread-2: while (!paymentProcessed) { }  // may loop forever

Six concepts — each explains a class of production concurrency bug in banking systems.

1. Heap

• Young/Old gen: Short-lived Payment DTOs in Eden; long-lived Account caches in Old — GC lesson covers collectors.
• Mutable shared state: Account balance on heap accessed by HTTP thread and async settlement worker — must coordinate.
• Immutability: Immutable Money record published after construction — safe to share without locks if reference published safely.
• False sharing: Two AtomicLong counters on same cache line — performance not correctness; use LongAdder.

2. Stack

• Thread confinement: Local BigDecimal amount on stack — no synchronization needed; each thread has own copy.
• Escape analysis: JVM may allocate non-escaping objects on stack (scalar replacement) — optimization transparent to JMM.
• Don't leak locals: Passing reference to local array to another thread after method returns — undefined if stack reused; use heap allocation.
• StackOverflowError: Infinite recursion in fee calculation — per-thread stack limit (~1MB default); not OutOfMemoryError on heap.

3. Metaspace

• Java 8+: Replaced PermGen — class metadata in native memory; grows until -XX:MaxMetaspaceSize.
• Classloader leaks: Redeploy WAR without releasing old ClassLoader — Metaspace grows until OOM on payment app restart cycles.
• Dynamic proxies: Spring @Transactional creates subclass per bean — monitor metaspace in long-running banking apps.
• Tuning: -XX:MaxMetaspaceSize=256m — fail fast; Metaspace GC triggers when class unloading possible.

4. Visibility

• Problem: Plain field write in one thread not guaranteed visible to another without synchronization.
• Symptom: Payment marked complete in DB but in-memory cache flag still false — mixed storage layers need consistent model.
• Fix options: synchronized, volatile, Atomic*, concurrent collection, or immutable replacement.
• Not a race on single write: Visibility is about seeing latest value; atomicity is about read-modify-write — both needed for balance debit.

5. Happens-before

• Program order: Actions in same thread appear in order to that thread — other threads may see reordering without hb.
• Monitor rules: Unlock happens-before subsequent lock on same monitor — basis of synchronized visibility.
• Thread start/join: start() happens-before run(); actions in thread happen-before join() returns.
• Safe publication: Final fields initialized in constructor happen-before read after this escape if reference published via hb (e.g. volatile or lock).

6. Volatile

• Status flags: volatile boolean settlementComplete — one writer, multiple readers; correct shutdown signal.
• Double-checked locking: Holder must be volatile (or use enum singleton) — pre-Java 5 broken without volatile.
• Not for balance: volatile long balance; balance -= amt is read-modify-write — still racy; use synchronized or AtomicLong.
• vs synchronized: volatile lighter for single field visibility; synchronized provides mutual exclusion plus visibility.

Code walkthrough

JMM demo — visibility failure, volatile fix, happens-before with synchronized:

• Plain boolean flag: Reconciliation loop may never see true — use volatile or AtomicBoolean for cross-thread status.
• join happens-before: Actions in worker thread visible to main after join() returns.
• AtomicLong: incrementAndGet is atomic and establishes happens-before with subsequent get — correct txn counter.
• synchronized: Unlock happens-before next lock — thread t2 guaranteed to see shared[0]=42.
• Single-core caveat: Visibility bugs may hide on laptop — test on multi-core CI.

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;

class PaymentProcessor {
    // BAD: plain boolean — visibility not guaranteed
    private boolean processedPlain = false;

    // GOOD: volatile — visibility for status flag
    private volatile boolean processedVolatile = false;

    private long balanceCents = 0; // needs lock or atomic for updates

    void markProcessedUnsafe() { processedPlain = true; }
    boolean isProcessedUnsafe() { return processedPlain; }

    void markProcessedSafe() { processedVolatile = true; }
    boolean isProcessedSafe() { return processedVolatile; }

    synchronized void credit(long cents) { balanceCents += cents; }
    synchronized long balance() { return balanceCents; }
}

class JmmDemo {
    public static void main(String[] args) throws Exception {
        PaymentProcessor proc = new PaymentProcessor();

        // Visibility demo — volatile vs plain (may not fail on single core!)
        Thread worker = new Thread(() -> {
            proc.credit(10_000L);
            proc.markProcessedSafe();
        });
        worker.start();
        worker.join(); // join happens-before main reads

        System.out.println("Balance after join: " + proc.balance());
        System.out.println("Processed (volatile): " + proc.isProcessedSafe());

        // AtomicLong for lock-free counter — visibility + atomicity
        AtomicLong txnCount = new AtomicLong(0);
        int threads = 8;
        CountDownLatch done = new CountDownLatch(threads);
        try (ExecutorService pool = Executors.newFixedThreadPool(threads)) {
            for (int i = 0; i < threads; i++) {
                pool.submit(() -> {
                    for (int j = 0; j < 1000; j++) txnCount.incrementAndGet();
                    done.countDown();
                });
            }
            done.await();
        }
        System.out.println("Txn count: " + txnCount.get() + " (expect 8000)");

        // Happens-before via synchronized
        Object lock = new Object();
        int[] shared = {0};
        Thread t1 = new Thread(() -> {
            synchronized (lock) { shared[0] = 42; }
        });
        Thread t2 = new Thread(() -> {
            synchronized (lock) { System.out.println("Seen: " + shared[0]); }
        });
        t1.start(); t1.join();
        t2.start(); t2.join();
    }
}

Production example

Spring Boot — JMM-aware payment cache and shutdown:

• volatile shutdown flag: K8s SIGTERM sets flag — worker loops exit cleanly without stale read.
• volatile reference publish: Swapping immutable SettlementBatch — readers see consistent snapshot.
• AtomicLong pendingCount: Cross-thread counter without synchronized bottleneck.
• Holder idiom: ClassLoader initializes Holder lazily — JVM guarantees safe publication of static final.

@Service
public class SettlementCoordinator {
    // volatile — visible to all settlement worker threads
    private volatile boolean shutdownRequested = false;

    private final AtomicLong pendingCount = new AtomicLong(0);

    // Immutable snapshot — safe publication after construction
    public record SettlementBatch(String batchId, List<String> paymentIds) {}

    private volatile SettlementBatch currentBatch; // reference swap is hb via volatile write

    @PreDestroy
    public void onShutdown() {
        shutdownRequested = true; // volatile write visible to workers
    }

    public void workerLoop() {
        while (!shutdownRequested) {
            SettlementBatch batch = currentBatch; // volatile read
            if (batch != null) processBatch(batch);
        }
    }

    public void publishBatch(SettlementBatch batch) {
        currentBatch = batch; // volatile write — workers see fully constructed batch
    }

    // WRONG: @Cacheable HashMap without ConcurrentHashMap — visibility + atomicity
    // RIGHT: Caffeine cache or ConcurrentHashMap for in-memory payment status
}

// Safe lazy init — initialization-on-demand holder (JMM correct)
class FxRateProvider {
    private static class Holder {
        static final FxRateProvider INSTANCE = new FxRateProvider();
    }
    static FxRateProvider getInstance() { return Holder.INSTANCE; }
}

Enterprise case study

Payment gateway in-memory dedup cache incident: A card processing service cached processed transaction IDs in a plain HashMap on a singleton bean — no synchronization. Under dual-active deployment, thread A on pod-1 wrote TXN-8842 as processed; thread B on pod-2 (separate JVM — separate heap, OK) wasn't the issue. Within the same pod, async notification thread and HTTP callback thread both accessed the HashMap — infinite loop risk aside, the dedup flag used a plain boolean processed field on a mutable DTO stored in the map. Callback thread never observed processed=true set by notification thread — duplicate chargebacks filed against merchant. Fix: ConcurrentHashMap for cache, AtomicBoolean or DB unique constraint for idempotency, removed reliance on cross-thread boolean visibility. Secondary: Metaspace OOM during same release from undeployed classloader leak in hot-reload staging — unrelated but discovered in same RCA week.

• Symptom: Duplicate chargebacks on 0.3% of async callback paths — only multi-threaded pod under load.
• Root cause: Plain field visibility + non-thread-safe HashMap structural modification.
• Fix: Idempotency key in DB (source of truth), ConcurrentHashMap, eliminate plain cross-thread flags.
• Lesson: JMM bugs don't throw — they duplicate money movement; test multi-threaded on CI.

Performance considerations

JMM and performance:

• volatile cost: Prevents some optimizations — fine for flags; don't mark every field volatile.
• synchronized contention: Visibility via locks — contention hurts throughput; narrow critical sections.
• Atomic vs lock: Low contention counters — AtomicLong/LongAdder cheaper than synchronized.
• False sharing: Padding or @Contended (JVM internal) for hot counters on same cache line.
• Immutable objects: No visibility issue for fields if object immutable and reference published safely — zero lock cost on reads.

Security considerations

JMM and security in payment systems:

• TOCTOU with stale reads: Authorization check reads stale role flag — privilege escalation window in multi-threaded handler.
• Unsafe publication: Escaping partially constructed Payment object — attacker thread reads uninitialized sensitive fields (theoretical with careful exploit).
• Timing side channels: Lock contention patterns leak transaction volume — niche threat in HSM integration.
• ThreadLocal credentials: Visibility within thread only — async breaks model; copy security context explicitly.

Scalability considerations

JMM at scale:

• Per-JVM heap: Horizontal scale doesn't share heap — use DB/Redis for cross-pod visibility, not static fields.
• Read-mostly immutable config: Fee tables loaded once, frozen — scales across threads without locks.
• Concurrent collections: ConcurrentHashMap segments reduce lock contention vs synchronized HashMap.
• Metaspace per pod: Same class set per replica — monitor metaspace uniformly across fleet.

Production challenges

Real JMM-related production failures:

• Heisenbugs: Visibility failure disappears when debugger attached — single-core dev machine masks bug.
• 32-bit long/double writes: Without volatile, long/double writes not atomic on 32-bit JVM — torn reads on amount (rare today).
• Metaspace death spiral: Classloader leak + frequent deploy — all pods OOM Metaspace during business hours.
• Assuming @Transactional flushes to other threads: DB commit visible to other pods; in-memory bean fields still need JMM rules.
• Double-checked locking regression: Copy-paste singleton without volatile — broken on ARM servers though works on x86 dev laptop.

Common mistakes

• Relying on plain boolean for cross-thread shutdown or status — use volatile or AtomicBoolean.
• Assuming assignment to long/double is atomic without volatile on 32-bit JVM — use volatile or AtomicLong.
• Using volatile for balance debit — visibility without atomicity; still racy.
• HashMap for cross-thread cache — use ConcurrentHashMap; structural modification races corrupt map.
• Ignoring happens-before when publishing objects — unsafe publication exposes partial construction.

Debugging guide

Debug JMM / visibility issues:

• Reproduce multi-core: Run stress test on 8+ core CI — not single-core laptop.
• Thread sanitizer alternatives: JCStress (OpenJDK) — formal concurrency tests for JMM claims.
• Code search: Grep for plain boolean/int flags on shared singleton beans accessed from multiple threads.
• Metaspace: jcmd <pid> VM.metaspace — track committed vs limit during deploy cycles.
• Symptom pattern: Intermittent wrong state, no exception — suspect visibility or race, not NPE.

# Metaspace stats
jcmd $(pgrep -f payments.jar) VM.metaspace

# JVM flags for debugging (dev only)
-XX:+TraceClassLoading
-XX:MaxMetaspaceSize=128m   # fail fast in staging

# JCStress (OpenJDK project)
// Test volatile visibility claim under all JVM optimizations

Best practices

• Use volatile for one-writer status flags (shutdown, initialized) — not for compound state updates.
• Prefer immutable objects + safe publication (final fields, volatile ref, or concurrent collection) over shared mutable fields.
• Use java.util.concurrent atomic classes and concurrent collections — they include correct JMM semantics.
• Establish happens-before explicitly — don't rely on "it works on my machine" timing luck.
• Set MaxMetaspaceSize in production — fail fast with clear error vs slow native OOM.
• Run concurrency stress tests on multi-core hardware in CI pipeline.
• DB is source of truth for payment idempotency — in-memory flags are optimization only with correct JMM.

Anti-patterns

• boolean done = false shared across threads without volatile — infinite loop or stale read.
• Double-checked locking without volatile instance field — broken publication on some architectures.
• Using Thread.yield() or sleep() to "flush" memory — not a JMM guarantee; use proper synchronization.
• Storing payment state in static mutable HashMap on @Service singleton — visibility and structural race.
• Assuming flush to DB makes in-memory copy visible to other threads in same JVM — it doesn't.

Staff engineer notes

• If the bug is intermittent, disappears under debugger, and involves shared flags — think JMM visibility first.
• volatile solves visibility for one field — it does not replace locking for check-then-act on account balance.
• Metaspace OOM during deploy windows is often classloader leak — not "need more memory" blindly.
• Happens-before is the vocabulary for code review — ask "what edge makes this write visible?"
• Safe publication of immutable Payment DTO is underrated — eliminates whole class of JMM bugs on read path.

Interview questions

Hands-on exercise

Lab: Java Memory Model visibility

• Run playground — observe AtomicLong counter reaching 8000 with 8 threads.
• Replace AtomicLong with plain long += in synchronized block vs unsynchronized — compare results.
• Test volatile shutdown flag — set from main after 1s, worker loop should exit.
• Remove volatile from processed flag — run on multi-core machine; note intermittent behavior.
• Implement holder idiom singleton FxRateProvider — verify lazy safe init.
• Bonus: run jcmd VM.metaspace on local Spring Boot app after startup.

Press Run to execute. Real javac + JVM via remote API. Falls back to simulator on network failure.

Architecture trade-offs

• volatile vs synchronized: volatile for single-field visibility; synchronized for atomic compound operations.
• Immutable vs mutable shared state: Immutable eliminates visibility bugs on fields; mutation needs coordination.
• Atomic vs lock: Atomics for simple counters; locks for multi-field invariants (debit+credit pair).
• In-memory cache vs DB: Cache fast but JMM/ cluster invalidation hard; DB is cross-pod truth.

Summary

The Java Memory Model defines when writes are visible across threads — the foundation behind synchronized, volatile, and java.util.concurrent. Map heap, stack, and Metaspace to your payment domain objects; establish happens-before for every cross-thread state transition; never use plain fields for async payment status. JMM literacy separates intermittent production money bugs from mystery "can't reproduce" tickets.

Key takeaways

• Heap is shared — stack is thread-confined; Metaspace holds class metadata.
• Visibility requires happens-before — plain field writes are not enough across threads.
• volatile gives visibility and ordering, not atomicity for read-modify-write.
• Use concurrent utilities and immutable objects — correct JMM semantics by design.
• Metaspace OOM and visibility Heisenbugs are production JMM failures — test multi-core, monitor metaspace.