Build Artifacts

Introduction

Build artifacts are immutable outputs of CI — container images, JARs, npm tarballs, Helm charts — stored in registries and promoted across environments by digest or version, never rebuilt per stage.

The story

Staging passed. Prod failed with "works on my machine." Diff showed staging ran npm run build at deploy time while prod used a three-day-old image tag latest that no longer matched any CI run. Root cause: no immutable artifact — teams promoted tags that floated. Fix: build once in CI, push ghcr.io/acme/api@sha256:abc…, promote that digest through staging → prod; deploy manifests pin digest, not tag.

Understanding the topic

The artifact is the contract between CI and CD. Immutable means the bits verified in CI are identical in prod — same layers, same dependencies, same compiled output.

• Immutable artifacts: content-addressed (SHA256 digest for OCI images); never overwrite in registry — new build = new digest.
• Registries: GHCR, ECR, GCR, Artifactory, Harbor — store images, Helm OCI, generic binaries with RBAC and retention policies.
• Digest promotion: staging deploys digest X; prod approval copies reference to digest X — no rebuild, no retag of mutable latest.
• Provenance: SLSA attestations, SBOM, build metadata (git SHA, pipeline run ID) attached to artifact.
• Artifact types: OCI image (most common), fat JAR/WAR, npm package to Verdaccia, Terraform bundle, static site tarball to S3.

Internal architecture

Build once, promote many — artifact flow from CI to environments:

Developer merge (SHA abc123)
   ↓
CI pipeline
   ├─ compile / test / scan
   ├─ docker build → image layers
   └─ push ghcr.io/acme/api:abc123
              └── digest sha256:9f3a…  ← immutable ID
   ↓
Artifact registry (RBAC: CI write, CD read)
   ↓
Staging deploy     → kubectl set image …@sha256:9f3a…
   ↓ smoke + integration
Prod promotion     → same digest (approval gate)
   ↓
Runtime            → identical bits to CI-verified artifact

Visual explanation

Two diagrams show where Build Artifacts lives in the delivery path and how teams implement it in production.

Step-by-step explanation

1. Identify deploy unit: container image, JAR, static bundle — one primary artifact per service.
2. Configure CI to build, scan (Trivy/Grype), and push to registry with tags git SHA and semver on release.
3. Capture digest after push: docker inspect --format='{{index .RepoDigests 0}}' or buildkit output.
4. Pass digest to CD via workflow output, artifact metadata file, or GitOps commit updating image digest field.
5. Ban rebuild-on-deploy in CD scripts — lint for docker build in deploy workflows.

Production implementation

GitHub Actions: build, push, output digest for GitOps promotion:

• Use OCI referrers for SBOM attachment (Cosign, ORAS).
• ECR: enable immutable tags; reject tag overwrite at registry policy level.
• Retention: lifecycle policy deletes untagged manifests after 30 days, keeps semver tags.

# .github/workflows/build-push.yml
name: Build and Push
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
      id-token: write
    outputs:
      digest: ${{ steps.build.outputs.digest }}
      image: ${{ steps.meta.outputs.tags }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - id: meta
        uses: docker/metadata-action@v5
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=sha,prefix=
            type=semver,pattern={{version}}
      - id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
      - name: Capture digest
        run: echo "digest=${{ steps.build.outputs.digest }}" >> $GITHUB_OUTPUT

  update-gitops:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { repository: org/gitops-repo, token: ${{ secrets.GITOPS_PAT }} }
      - run: |
          yq -i '.image.digest = "${{ needs.build.outputs.digest }}"' \
            apps/api/overlays/staging/kustomization.yaml
          git commit -am "promote api ${{ needs.build.outputs.digest }} to staging"
          git push

Execution workflow

Real-world use

The 12-factor app principle "strict separation of build and run stages" predates containers but OCI registries made it enforceable. Supply-chain attacks (SolarWinds, codecov) drove SLSA and Sigstore Cosign adoption — artifacts must be signed and verifiable at deploy admission.

Enterprise use cases

Netflix Spinnaker pipelines promote immutable AMIs and Docker images by ID. Google Bazel builds produce hermetic artifacts stored in CAS. Banks use Artifactory with promotion paths: dev-local → libs-release-local → prod-remote — physical repo move, not rebuild.

• Multi-arch images: build matrix amd64/arm64; manifest list digest is promotion unit.
• Helm OCI: chart pushed as OCI artifact; deploy pins chart digest alongside image digest.
• Lambda/container: same digest model — ECR image URI with @sha256 in SAM/CDK.

Production case study

A healthtech SaaS moved from "SSH + docker build on prod box" to CI-built images in ECR with digest-pinned Kubernetes manifests. SOC2 auditors required proof that prod image matched CI scan results.

• Before: prod builds differed from CI; Trivy scan only in dev pipeline.
• After: single build job; digest in GitOps; Kyverno rejects unsigned or unscanned images.
• Evidence: workflow run links git SHA → digest → deploy manifest commit chain.
• Outcome: passed audit; incident rollback became kubectl rollout undo or GitOps revert to prior digest.

Trade-offs

• Digest-only deploys: maximum reproducibility; humans can't read digest — pair with semver tag metadata.
• Mutable tags (latest): convenient for dev; catastrophic for prod traceability — never promote latest.
• Per-env rebuild: allows env-specific compile flags; breaks "what we tested is what we ship."
• Large artifact retention: audit/compliance win; storage cost grows — lifecycle policies required.

Security implications

Artifacts are the supply-chain payload — compromise at build or registry equals compromise in prod.

• Sign images with Cosign; admission controller (Kyverno, Gatekeeper) verifies signature before pull.
• Scan at push and block critical CVEs; rescan on new CVE data (registry webhook re-trigger).
• Registry RBAC: CI service account write-only to repo; prod cluster read-only; no human push to prod repos.
• Prevent artifact hijack: pin base images by digest in Dockerfile; dependabot updates digest deliberately.

Scalability analysis

Registries and artifact volume become bottlenecks at high deploy frequency.

• Geo-replicate registries (ECR replication, Harbor replication) for multi-region pull latency.
• Layer deduplication across services reduces storage — shared base images matter.
• Manifest lists for multi-arch double push volume — plan registry quota.
• GitOps repos updating per-deploy create commit noise — use digest bot with batching or PR automation.

Staff engineer insights

• If CD runs docker build, you don't have CD — you have remote compilation with extra steps.
• Promotion is a metadata operation on digest, not a pipeline rerun with APP_ENV=prod.
• Tag semver for humans, digest for machines — both can reference same manifest.
• Artifact retention policy is a legal requirement in finance — plan before registry bill shocks finance.

Best practices

• Pin base images and dependencies by digest in Dockerfiles where feasible.
• Attach git SHA, pipeline URL, and SBOM as OCI labels or attestations.
• Use registry immutable tag policies for release tags.
• Deploy with image@sha256:… in Kubernetes, not image:v1.2.3 alone.

Common mistakes

• Promoting semver tag while registry tag was overwritten — always verify digest.
• Building different Dockerfile targets per env (dev vs prod) — drift in dependencies.
• Storing secrets in image layers — use runtime secret injection, scan with trufflehog.

Advanced interview questions

Hands-on exercise

Build and push an image from CI, capture digest, and deploy to local kind cluster by digest — not tag.

• Verify running pod imageID matches pushed digest in kubectl describe.
• Change code without rebuilding — confirm old digest still runs until new push.
• Document promotion flow in ARTIFACTS.md.

docker build -t demo:local .
docker push ghcr.io/YOUR_USER/demo:$(git rev-parse --short HEAD)
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ghcr.io/YOUR_USER/demo:$(git rev-parse --short HEAD))
echo "Deploying $DIGEST"
kubectl set image deployment/demo app=$DIGEST
kubectl rollout status deployment/demo

Summary

You understand immutable artifacts, registries, and digest promotion — the mechanism that ensures staging verification applies to production. Build in CI, pin by digest, sign and scan, promote metadata not pipelines.