Jenkins Basics

Introduction

Jenkins is the original widely-adopted open-source automation server — still running critical pipelines at enterprises worldwide. This lesson covers the controller/agent architecture, Jenkinsfile pipeline-as-code, and the plugin ecosystem staff engineers must navigate for maintenance, security, and migration planning.

The story

A bank's payment platform still ran on Jenkins 2.263 with 340 plugins — half abandoned. A Log4j CVE in a plugin nobody remembered installing blocked all releases for 72 hours. The post-incident plan wasn't "migrate tomorrow" but documenting controller/agent boundaries, pinning plugins, moving new services to GitHub Actions, and strangling Jenkins workload over 18 months without stopping Friday releases.

Understanding the topic

Jenkins architecture primitives:

• Controller (master): web UI, job scheduling, plugin runtime, credentials store. Should NOT run build workloads — CPU/memory contention and security isolation.
• Agents (nodes): executors that run Pipeline steps. Static VMs, Docker clouds, or Kubernetes pod templates. Labels route jobs (agent { label 'linux-docker' }).
• Jenkinsfile: Declarative or Scripted Pipeline in repo root — pipeline { agent any; stages { ... } }. Multibranch scans branches automatically.
• Plugins: Jenkins extensibility model — Git, Docker, Kubernetes, credentials-binding, Blue Ocean. Each plugin is a supply-chain dependency.

Internal architecture

Controller → agent execution model

Jenkins Controller (scheduling, UI, plugins)
   ↓ assigns
Agent pool (VM / Docker / K8s pod)
   ↓ executes
Jenkinsfile stages (checkout, build, test, deploy)
   ↓ uses
Credentials binding + shared libraries

Visual explanation

Two diagrams show where Jenkins Basics lives in the delivery path and how teams implement it in production.

Step-by-step explanation

1. Developer commits Jenkinsfile to repo — Multibranch Pipeline job scans SCM, creates per-branch jobs automatically.
2. Controller parses Declarative Pipeline, allocates an agent matching agent block constraints.
3. Agent checks out source, runs stage steps (shell, bat, docker, withCredentials). Each stage can post conditions (post { failure { ... } }).
4. Plugins provide steps — docker.build(), kubernetesDeploy, junit. Shared libraries factor Groovy utilities across repos.
5. Logs and artifacts return to controller UI; Blue Ocean or Pipeline Stage View shows timing; failed builds notify via email/Slack plugin.

Production implementation

Declarative Jenkinsfile — agent label, stages, credentials, post:

pipeline {
  agent { label 'linux-docker' }
  options {
    timeout(time: 30, unit: 'MINUTES')
    disableConcurrentBuilds()
    buildDiscarder(logRotator(numToKeepStr: '20'))
  }
  environment {
    APP_NAME = 'payments-api'
  }
  stages {
    stage('Checkout') {
      steps { checkout scm }
    }
    stage('Test') {
      steps {
        sh 'npm ci && npm test'
      }
      post {
        always {
          junit 'reports/junit.xml'
        }
      }
    }
    stage('Build Image') {
      steps {
        script {
          docker.build("${APP_NAME}:${env.GIT_COMMIT}")
        }
      }
    }
    stage('Deploy Staging') {
      when { branch 'main' }
      steps {
        withCredentials([string(credentialsId: 'staging-kubeconfig', variable: 'KUBECONFIG')]) {
          sh './scripts/deploy.sh staging'
        }
      }
    }
  }
  post {
    failure {
      slackSend channel: '#ci-alerts', message: "Build failed: ${env.BUILD_URL}"
    }
  }
}

Execution workflow

Real-world use

undefined

Enterprise use cases

Shared library for DRY deploy logic — @Library('platform-pipeline@v2') _:

Production case study

Case study: Telecom operator stabilized 2000-job Jenkins estate.

• Problem: single controller, 12-min queue times, plugin conflicts blocked upgrades 2 years.
• Fix: controller HA pair; K8s dynamic agents; plugin allowlist (180→95); Jenkinsfile linter in PR.
• Strangler: new microservices on GitLab CI; Jenkins retained for mainframe batch jobs only.
• Outcome: queue p95 12 min → 90 sec; zero plugin-CVE release freezes in following year.

Trade-offs

• Jenkins pros: infinite plugin flexibility; air-gapped friendly; teams already know Groovy pipelines.
• Jenkins cons: plugin CVEs; controller SPOF without HA; Groovy shared libraries untestable without harness.
• Declarative vs Scripted: Declarative enforces structure — prefer for code review; Scripted for dynamic logic only.
• Static agents vs K8s: K8s ephemeral pods improve isolation; static VMs accumulate state.

Security implications

Jenkins is a high-value attack target — controller holds credentials for everything:

• Controller isolation: no inbound agent→controller trust beyond JNLP/WebSocket; agents in DMZ, controller in secure zone.
• Credentials plugin: domain-scoped credentials; never bind prod creds to MR jobs from forks.
• Script approval: sandbox untrusted Pipeline Groovy; shared library changes require admin review.
• Plugin minimization: monthly audit; remove unused; prefer CloudBees tested plugin set.
• Migration off static AWS keys: use IAM OIDC plugin or external secrets (Vault, AWS Secrets Manager).

Scalability analysis

Scaling Jenkins without controller meltdown:

• Executor count: controller UI degrades above ~500 executors — federate with multiple controllers or migrate workloads.
• Kubernetes plugin: pod templates per pipeline type; idle timeout reaps pods; resource limits per container.
• Shared library caching: library version pinned; @Library('x@main') is production roulette.
• Build queue: priority sorter plugin for hotfix lanes; throttle parallel deploy stages.
• Artifact storage: offload to S3/Nexus — controller disk is not an artifact registry.

Staff engineer insights

• If your Jenkins controller runs builds, you have two problems — fix that before optimizing anything else.
• Shared libraries are code — apply PR review, semver tags, and integration tests with Jenkins Pipeline Unit.
• Plan Jenkins migration as strangler fig, not big-bang — business cannot pause releases for 6 months.
• Document every plugin's owner and last-used date — orphan plugins become CVE roulette.

Best practices

• Use Declarative Pipeline unless dynamic Groovy is unavoidable.
• Pin agent labels — `agent any` lands builds on wrong OS.
• Set global build timeouts and log rotation — disk fills silently.
• Store artifacts in external blob storage, not controller `$JENKINS_HOME`.
• Use Configuration as Code (JCasC) for controller settings in git.

Common mistakes

• Installing "just one more plugin" without CVE check — average Jenkins has 100+ plugins, 30% unmaintained.
• Shared library @main — breaking change takes down all pipelines instantly.
• Running Docker-in-Docker on controller executors — privilege escalation risk.
• Credential binding in shared library without folder-scoped RBAC — any job reads prod keys.

Advanced interview questions

Hands-on exercise

Exercise: Convert this freestyle job concept to a Declarative Jenkinsfile with test, docker build, and conditional deploy on main.

# Freestyle job (ClickOps) — document as Jenkinsfile
# 1. git clone from origin/main
# 2. mvn test
# 3. docker build -t app:$BUILD_NUMBER .
# 4. if branch main: kubectl apply -f k8s/

Summary

You understand Jenkins controller/agent architecture, Jenkinsfile pipelines, and the plugin ecosystem — enough to operate legacy estates, harden security, and plan modern CI migration without stopping releases.