Pipelines

Introduction

Pipelining sends multiple commands without waiting for each reply — one round trip for hundreds of GETs instead of hundreds of RTTs. Amazon catalog pages batch MGET and pipelined HGETALL across SKUs; this is often the largest client-side win after choosing Redis.

Pipeline is not atomic — other clients' commands can interleave between yours unless you use MULTI or Lua. Lettuce and Spring executePipelined wrap native pipelining.

Size batches reasonably — megabyte pipelines stress client and server buffers.

Understanding the topic

Key concepts

• Send N commands; read N replies after flush.
• RTT amortization — dominant win on WAN.
• Not transactional — no atomicity.
• Works with all command types.
• Spring RedisCallback executePipelined.
• Combine with MGET/HMGET when same command repeats.

sequenceDiagram
  Client->>Redis: CMD1 CMD2 CMD3
  Redis->>Redis: execute sequentially
  Redis->>Client: RSP1 RSP2 RSP3

Step-by-step explanation

1. Client buffers commands in output buffer.
2. Flush sends entire batch to socket.
3. Server processes commands sequentially on event loop.
4. Replies buffered and returned in order.
5. Client reads all replies matching send order.

Syntax reference

Common commands

• Pipeline ≠ MULTI.
• Match command count to reply count.
• Use for read-heavy batch APIs.

# redis-cli --pipe for bulk import
# Interactive: echo -e "GET a\nGET b\nGET c" | redis-cli --pipe

# Compare: 1000 sequential GETs = 1000 RTT
# 1 pipeline of 1000 GETs = 1 RTT + processing

Informative example

Pipeline 500 product cache reads in Spring Boot 3:

List<Object> prices = redis.executePipelined((RedisCallback<Object>) conn -> {
    for (String sku : skus) {
        conn.stringCommands().get(("price:" + sku).getBytes(StandardCharsets.UTF_8));
    }
    return null;
});

Cast results carefully — order matches send order. For identical command use MGET when keys fit one slot/cluster node.

Real-world use

Real-world use cases

• Product page hydrate 50 SKUs.
• Bulk session validation HMGET pipeline.
• Metrics flush batch INCR pipeline.
• Migration import via redis-cli --pipe.
• Cache warming on deploy.

Best practices

• Pipeline hot read paths first.
• Batch size 100–1000 depending on value size.
• Use MGET when all commands are GET same type.
• Monitor client timeout on large pipelines.
• Co-locate app and Redis to minimize RTT anyway.
• Profile before pipelining serial code.

Common mistakes

• Confusing pipeline with transaction atomicity.
• Unbounded pipeline size OOM client buffer.
• Ignoring reply order mapping to requests.
• Pipelining across cluster slots without grouping by node.

Advanced interview questions

Summary

Pipeline = batch RTT, biggest client optimization.