GitLab CI Pipelines

Introduction

GitLab CI/CD embeds pipelines in the same platform as source control, issues, and container registry. Pipelines are defined in .gitlab-ci.yml using stages, includes for composition, rules for conditional execution, and child pipelines for monorepo scale — patterns GitLab-native teams use for full DevSecOps lifecycles.

The story

A monorepo with 800 microservices ran a single 4-hour pipeline on every commit until platform engineers split it with child pipelines triggered per changed path. Includes pulled shared templates from a central project; rules skipped docs-only commits. Median feedback dropped to 11 minutes — without losing the single-repo developer experience product wanted.

Understanding the topic

GitLab CI core constructs:

• Stages: ordered pipeline phases (build → test → deploy). Jobs in the same stage run in parallel; next stage waits for all prior jobs to succeed (unless allow_failure).
• Includes: compose YAML from local files, remote projects, or templates — include: { project: 'platform/ci-templates', file: '/node/test.yml' }.
• Rules: replace legacy only/except — expressive if: conditions on branch, MR, variables, paths. First matching rule wins; default when: never if none match.
• Child pipelines: trigger: job generates a dependent pipeline (dynamic or included YAML) — isolates monorepo subsystems and enables parent/child artifact passing.

Internal architecture

GitLab pipeline hierarchy

.gitlab-ci.yml (include templates)
   ↓
Stages: .pre → build → test → deploy → .post
   ↓
Jobs (rules filter which run)
   ↓
Child pipeline trigger (monorepo / dynamic)
   ↓
GitLab Runner executes on shell/docker/k8s executor

Visual explanation

Two diagrams show where GitLab CI Pipelines lives in the delivery path and how teams implement it in production.

Step-by-step explanation

1. Developer pushes or opens MR — GitLab loads root .gitlab-ci.yml and merges all include files into one pipeline definition.
2. GitLab evaluates each job's rules: block top-to-bottom. Matching jobs enter the pipeline assigned to their stage.
3. GitLab Runner (shared or self-hosted) picks jobs from the queue, pulls the Docker image (image: key), and runs script: sections.
4. Artifacts (artifacts:) and cache (cache:) pass between jobs/stages. needs: creates DAG edges bypassing stage order when required.
5. A trigger job launches a child pipeline with its own YAML — parent waits unless strategy: depend configured otherwise.

Production implementation

Production .gitlab-ci.yml with includes, rules, stages, and child pipeline:

include:
  - project: 'platform/ci-templates'
    ref: v2.4.0
    file:
      - '/templates/node-test.yml'
      - '/templates/docker-build.yml'

stages:
  - build
  - test
  - deploy

variables:
  DOCKER_TLS_CERTDIR: "/certs"

build-image:
  stage: build
  extends: .docker-build
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - src/**/*
        - Dockerfile
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

unit-test:
  stage: test
  extends: .node-test
  needs: [build-image]
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy-staging:
  stage: deploy
  environment:
    name: staging
    url: https://staging.example.com
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - helm upgrade --install app ./chart --set image.tag=$CI_COMMIT_SHA

trigger-service-pipeline:
  stage: build
  rules:
    - changes:
        - services/payments/**/*
  trigger:
    include: services/payments/.gitlab-ci.yml
    strategy: depend

Execution workflow

Real-world use

undefined

Enterprise use cases

Shared template in platform/ci-templates — consumed via include:

Production case study

Case study: Insurance company migrated 200 Jenkins jobs to GitLab CI includes.

• Challenge: Jenkins Groovy untestable; plugin CVEs; no MR-native pipeline view.
• Approach: 12 include templates (build, scan, deploy); child pipelines per business unit monorepo folder.
• Rules: SAST template mandatory via include — teams cannot remove security stage.
• Outcome: pipeline config in git for 100% of projects; audit trail satisfied FFIEC reviewers.

Trade-offs

• Includes pros: DRY templates across groups; central security updates propagate instantly.
• Includes cons: debugging merged YAML requires CI/CD → Editor → View merged; circular includes fail cryptically.
• Child pipelines pros: monorepo isolation; parallel subsystem releases; dynamic generated pipelines.
• Child pipelines cons: parent visibility fragmented; artifact passing requires explicit configuration.
• Rules vs only/except: rules are verbose but unambiguous — only/except deprecated for good reason.

Security implications

GitLab CI security model:

• Protected branches + protected variables: prod secrets only exposed on protected branches/MRs from protected sources.
• Masked variables: prevent echo to job logs — regex must match entire value.
• CI_JOB_TOKEN: scoped API access — restrict cross-project access in group settings.
• Runner isolation: shared runners are multi-tenant — sensitive builds need dedicated runners.
• Include from trusted projects only: remote include is arbitrary code execution in pipeline context.

Scalability analysis

Scaling GitLab CI across large orgs:

• Runner autoscaling: GitLab Runner on K8s with horizontal pod autoscaler matches queue depth.
• Child pipelines for monorepos: avoid 500-job parent — trigger per changed component.
• Cache strategy: cache:key:files on lockfiles prevents stale cross-branch pollution.
• Merge trains: serialized MR pipelines on busy default branches reduce semantic conflicts.
• Pipeline efficiency: rules:changes with path filters skip 70%+ jobs on docs-only commits.

Staff engineer insights

• Always inspect merged YAML in GitLab UI before blaming "CI didn't run" — rules probably excluded the job.
• Child pipelines are monorepo medicine, not default — start with rules:changes before adding trigger complexity.
• Pin include refs to tags, not main — template drift breaks hundreds of pipelines at once.
• Merge trains are underrated for high-velocity default branches — use when >20 MRs/day hit main.

Best practices

• Use `extends` + hidden jobs (.` prefix) for composition within a repo.
• Set `default: retry: 2` for transient infra failures on network-heavy jobs.
• Publish junit and coverage reports as artifacts — GitLab MR widget consumes them.
• Use `environment: auto_stop_in` for review apps — cost control.
• Validate YAML with `gitlab-ci-lint` API in a pre-commit hook.

Common mistakes

• rules:changes on MRs compares to target branch — misunderstanding causes "works on branch, fails on MR".
• Child pipeline variables do not inherit automatically — pass explicitly with `forward: pipeline_variables`.
• Global cache key — causes cross-project leakage on shared runners.
• Using `only: [tags]` alongside rules — only/except and rules conflict unpredictably.

Advanced interview questions

Hands-on exercise

Exercise: Write rules so `deploy-prod` runs only on tags matching `v*.*.*` from protected branches, and `lint` skips when only `*.md` files change.

stages: [lint, deploy]

lint:
  stage: lint
  script: npm run lint
  # TODO: add rules

deploy-prod:
  stage: deploy
  script: ./deploy.sh production
  # TODO: add rules + environment

Summary

You understand GitLab CI pipelines: stages for ordering, includes for template reuse, rules for conditional execution, and child pipelines for monorepo decomposition — the constructs behind GitLab-native DevSecOps.