Azure DevOps Pipelines

Introduction

Azure DevOps Pipelines provides YAML-defined CI/CD deeply integrated with Azure, GitHub, and enterprise identity (Entra ID). This lesson covers pipeline YAML structure, service connections for external targets, environments with deployment history, and approval gates — the patterns Microsoft-centric orgs use for regulated delivery.

The story

A healthcare SaaS on Azure needed SOC2 evidence that every production deploy had human approval and traceable artifact lineage. Azure DevOps environments with required approvers, service connections scoped to subscription RBAC, and YAML multi-stage pipelines gave auditors a single dashboard — replacing email-based "approved by Bob" spreadsheets.

Understanding the topic

Azure Pipelines key concepts:

• YAML pipelines: azure-pipelines.yml with trigger, stages, jobs, steps. Template extends (extends: / template:) compose org standards.
• Service connections: authenticated links to Azure RM, ACR, Kubernetes, GitHub, AWS. Backed by workload identity federation (OIDC) or service principals — scoped to resource groups.
• Environments: named deployment targets (staging, production) with resource tracking (AKS, App Service), deployment history, and checks/approvals.
• Approvals: pre-deployment checks on environment — manual approval, Azure Policy, template compliance, invoke REST gate.

Internal architecture

Azure DevOps pipeline flow

azure-pipelines.yml (trigger / PR)
   ↓
Stages: Build → Test → DeployStaging → DeployProd
   ↓
Service connection (OIDC → Azure RM)
   ↓
Environment: production (approval gate)
   ↓
Azure resource (AKS / App Service / Function)

Visual explanation

Two diagrams show where Azure DevOps Pipelines lives in the delivery path and how teams implement it in production.

Step-by-step explanation

1. Commit triggers pipeline — Azure DevOps evaluates trigger/PR paths and selects default or branch-specific YAML.
2. Build stage runs on Microsoft-hosted or self-hosted agents (pool: vmImage: 'ubuntu-latest').
3. Pipeline publishes artifacts (PublishPipelineArtifact) or container images to ACR via service connection.
4. Deployment job targets environment: production — Azure DevOps pauses for configured approvals/checks.
5. Approved deploy uses service connection's workload identity to apply Bicep/Helm/Azure CLI to target subscription.

Production implementation

Multi-stage Azure Pipeline with service connection and environment approval:

trigger:
  branches:
    include: [main]

variables:
  imageRepository: 'payments-api'
  containerRegistry: 'myregistry.azurecr.io'
  tag: '$(Build.BuildId)'

stages:
- stage: Build
  jobs:
  - job: BuildAndPush
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: Docker@2
      displayName: Build and push
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: '**/Dockerfile'
        containerRegistry: 'my-acr-service-connection'
        tags: |
          $(tag)
          latest

- stage: DeployStaging
  dependsOn: Build
  jobs:
  - deployment: DeployStaging
    pool:
      vmImage: 'ubuntu-latest'
    environment: staging
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureWebAppContainer@1
            inputs:
              azureSubscription: 'my-azure-rm-oidc'
              appName: 'payments-staging'
              containers: '$(containerRegistry)/$(imageRepository):$(tag)'

- stage: DeployProduction
  dependsOn: DeployStaging
  jobs:
  - deployment: DeployProduction
    environment: production   # configure approvers in Environments UI
    strategy:
      runOnce:
        deploy:
          steps:
          - task: AzureWebAppContainer@1
            inputs:
              azureSubscription: 'my-azure-rm-oidc'
              appName: 'payments-prod'
              containers: '$(containerRegistry)/$(imageRepository):$(tag)'

Execution workflow

Real-world use

undefined

Enterprise use cases

Workload identity federation service connection — no client secret in Azure DevOps:

Production case study

Case study: Azure ISV achieved SOC2 Type II with pipeline-enforced approvals.

• Requirement: dual approval for prod; deploy artifact must match signed build from main.
• Implementation: environments with 2 required approvers + artifact source branch check; OIDC to prod subscription.
• Evidence: Azure DevOps deployment history exported to SIEM — who approved, which commit, which artifact.
• Outcome: audit finding closed in one quarter; deploy frequency unchanged at 3× daily.

Trade-offs

• Azure-native pros: seamless Entra ID RBAC, ARM/Bicep deploy tasks, environment audit trail.
• Azure-native cons: less ecosystem flexibility outside Microsoft stack; YAML template syntax learning curve.
• Classic vs YAML: classic release pipelines deprecated — YAML multi-stage is standard.
• Microsoft-hosted agents: convenient but egress costs to on-prem; self-hosted for hybrid.

Security implications

Service connections are the crown jewels:

• OIDC federation: prefer workload identity over service principal client secrets — no rotation toil.
• Scope connections: one connection per subscription/environment — not one god connection for all Azure.
• Pipeline permissions: restrict which YAML files can use prod service connection via environment checks.
• Secret variables: mark sensitive; never print in bash — Azure masks ADO secret vars automatically.
• Open access pipelines: disable for prod environments — only designated pipelines deploy production.

Scalability analysis

Enterprise Azure DevOps scale patterns:

• Template repositories: org-level YAML templates versioned in git — propagate security baselines.
• Parallel jobs: Microsoft-hosted parallel job count licensed per org — plan matrix fan-out.
• Self-hosted agent pools: VMSS autoscaling for VNet-integrated deploys to private AKS.
• Artifact retention: configure retention policies — default storage grows unbounded.
• Multi-project federation: service connections don't cross projects — duplicate with RBAC per product line.

Staff engineer insights

• Deployment jobs (not regular jobs) populate environment deployment history — use them even for CLI deploys.
• Service connection sprawl is safer than one mega-connection — blast radius beats convenience.
• Approvals on environment beat manual validation tasks in YAML — auditors understand environment gates.
• If you're GitHub-primary but Azure-primary cloud, Azure DevOps Environments + GitHub Actions OIDC is valid hybrid.

Best practices

• Use deployment jobs with `environment:` for all prod-targeting steps.
• Store pipeline templates in a dedicated git repo with semver tags.
• Enable "Protect branching" on main — build validation required.
• Use `@self` template references for repo-local templates; org templates via git resource.
• Export deployment logs to Log Analytics for long-term audit retention.

Common mistakes

• Regular job deploying to prod — no deployment history, no approval gate attachment.
• Service principal secret expiring silently — OIDC federation avoids this class of outage.
• Environment named 'production' in YAML but not created in project — deploy succeeds without intended checks.
• Mixing classic release pipelines with YAML — duplicate logic, divergent behavior.

Advanced interview questions

Hands-on exercise

Exercise: Add a DeployProduction stage with environment approval and an AzureCLI deploy step using service connection `payments-prod-oidc`.

stages:
- stage: Build
  jobs:
  - job: Build
    steps:
    - script: echo "Build complete"

Summary

You understand Azure DevOps Pipelines: YAML multi-stage structure, service connections with OIDC, environments with deployment history, and approval gates — the Microsoft-stack CI/CD model for regulated enterprise delivery.