Layers & Build Cache

Introduction

Each Dockerfile instruction creates a cacheable layer. If nothing above changed, Docker reuses the cached layer — making rebuilds blazingly fast. Misordering instructions destroys the cache.

Purpose of this lesson

This lesson teaches Layers & Build Cache as an engineering decision: what problem it solves, when to use it, how to implement it safely, and what signals tell you it is failing.

Understanding the topic

Use this whenever you create or review an application image. Build quality affects deploy speed, security exposure, reproducibility, CI cost, and how quickly engineers can diagnose failures.

Core concepts to understand:

• Cache key = previous layer + the exact instruction + input files.
• Change a source file? All COPY . . layers below invalidate.
• RUN apt-get update && apt-get install -y … must be one layer.
• BuildKit improves caching with mount-based caches (e.g. for npm/maven).

Visual explanation

Architecture or command flow to keep in mind:

# Good — deps cache survives source edits
COPY package*.json ./
RUN npm ci          # cached as long as package*.json doesn't change
COPY . .            # only this layer rebuilds on code changes

# Bad — source change invalidates deps install
COPY . .
RUN npm ci

Step-by-step explanation

1. Start from a trusted, pinned base image and document why it fits the runtime.
2. Order Dockerfile instructions from least-changing dependency metadata to most-changing source files so the cache stays useful.
3. Build locally with BuildKit, inspect layers with docker history, and run the image with the same command CI will use.
4. Test shutdown behavior, health checks, non-root execution, and runtime configuration before pushing the image.
5. Tag with an immutable version or git SHA, scan the image, and promote the same artifact through environments.

Informative example

Use the example below as a working baseline, then verify the runtime behavior instead of assuming the command or file is correct.

# Good — deps cache survives source edits
COPY package*.json ./
RUN npm ci          # cached as long as package*.json doesn't change
COPY . .            # only this layer rebuilds on code changes

# Bad — source change invalidates deps install
COPY . .
RUN npm ci

A production-minded check usually includes docker ps, docker logs, docker inspect, and one validation from outside the container such as curl, a database connection, or a registry pull.

# Verification loop for Layers & Build Cache
docker ps -a
docker logs --tail=100 <container-name>
docker inspect <container-or-image-name>
docker system df

Real-world use

Pinned dependency files + correct ordering can shrink a 90s CI build to 8s on rebuilds — the single biggest dev-velocity win in Docker.

Enterprise use cases

In a mature engineering organization, Layers & Build Cache is documented as a repeatable pattern with approved base images, ownership labels, CI checks, security expectations, rollback notes, and troubleshooting commands. The difference between a tutorial and production practice is that every container decision must be observable, reviewable, and reversible.

Best practices

• Copy dependency manifests first, install, then copy source.
• Combine related apt-get commands into one RUN.
• Use BuildKit cache mounts: RUN --mount=type=cache,target=/root/.npm npm ci.

Common mistakes

• apt-get install without apt-get update in the same layer — uses a stale cache.

Debugging tips

• Use docker build --progress=plain when BuildKit output hides the failing build step.
• Run the image with a temporary shell or overridden command to inspect files, users, environment, and entrypoint behavior.
• Check docker history for accidental secrets, unexpectedly large layers, and cache-busting instructions.

Optimization strategies

• Copy dependency lock files before source files so dependency installation remains cached during code edits.
• Use multi-stage builds to keep compilers, package managers, test tools, and source maps out of runtime images.
• Adopt BuildKit cache mounts for npm, Maven, pip, Go, or cargo dependencies in CI.

Advanced interview questions

Hands-on exercise

Create a small lab for Layers & Build Cache: run the example, inspect the created Docker object, intentionally introduce one mistake, and record the command that reveals the failure. The goal is not just to make the happy path work; it is to build operational reflexes.

# Hands-on lab scaffold
mkdir -p docker-layers-build-cache-lab
cd docker-layers-build-cache-lab

# Add the Dockerfile, compose.yml, or command from this lesson.
# Then run one happy-path test and one broken-path test.
docker version
docker info
docker system df

Summary

Layers & Build Cache matters because Docker is not only a packaging tool; it is a runtime, build, networking, storage, and delivery workflow. Treat each lesson as a production habit: make it repeatable, inspectable, secure, and easy to debug.