DevOps Principles and AWS Delivery Architecture

AWS Cloud & DevOps Engineering · Lesson 24 · DevOps Automation and IaC

Lesson Overview

Module 2 taught you how to run TechLearningPro workloads on AWS: APIs, containers, events, secrets, and Step Functions sagas. None of that survives Monday enrollment week if the only way to ship a change is a console click and a Slack message that says “I think prod is updated.” DevOps is not a job title and not a tool vendor logo. It is the discipline of shortening the path from a reviewed idea to a safe, observable change in production—and shortening the path from production signal back to the next improvement.

This lesson opens Module 3 by connecting culture, feedback loops, automation, trunk-based delivery habits, and deployment strategy thinking to a concrete AWS delivery architecture: build → test → deploy → observe. You will map a TechLearningPro value stream for the enrollment API and the infrastructure that hosts it. You will see where humans belong (approvals, design review) and where humans do not belong (hand-editing prod security groups at 1 a.m.). You will leave with a topology you can implement with AWS CodeBuild and CodePipeline or GitHub Actions for AWS Deployments, and with the Git governance that Git Workflows and Repository Governance will harden.

Cross-course primers exist if you need them: /learn/git for branching hygiene and /learn/ci-cd-automation for pipeline concepts in the abstract. This course does not clone those tracks. Here the lens is AWS accounts, IAM roles, artifacts in S3 or ECR, environment promotion, budget alerts, and cleanup of every lab pipeline you create.

Learning Objectives

By the end of this lesson you will be able to:

1. Define DevOps as culture plus practices plus automation—not as “the team that runs Jenkins.”
2. Explain the build → test → deploy → observe loop for TechLearningPro delivery on AWS.
3. Map a value stream from commit to production signal for an enrollment or order change.
4. Distinguish continuous integration, continuous delivery, and continuous deployment with honest trade-offs.
5. Relate trunk-based development habits to safer AWS promotions without requiring every team to use the same branching fashion.
6. Place quality gates (unit, integration, security, policy) before environment promotion.
7. Sketch a multi-account or multi-environment AWS delivery topology with clear blast-radius boundaries.
8. Choose when console changes are emergencies versus when they are process failures.
9. Connect deployment strategies (rolling, blue/green, canary) to observability and rollback at a conceptual level—detail comes in lesson 34.
10. Design feedback loops: metrics, logs, traces, and incident learnings that re-enter the backlog.
11. Apply least-privilege thinking to pipeline roles before you write buildspecs.
12. Budget, tag, and clean up delivery lab resources (artifact buckets, unused pipelines, leftover build projects).
13. Write a short ADR for TechLearningPro’s delivery architecture choices.
14. Preview how Git governance, CI/CD fundamentals, CodePipeline, GitHub Actions OIDC, and CloudFormation fit the same topology.

Prerequisites

• Module 2 application platform fluency: how TechLearningPro serves APIs, containers, and serverless workflows in AWS.
• AWS Identity and Access Management: roles and least privilege—pipelines will assume roles.
• AWS Account Setup, IAM Identity Center and Organizations: why separate accounts exist for blast radius.
• AWS Well-Architected Framework and Cost Basics: operational excellence and cost awareness.
• Optional primers: /learn/git, /learn/ci-cd-automation—concepts only; AWS delivery specifics live here.
• A sandbox account with a budget alert before you create artifact stores or pipeline resources in later labs.
• Willingness to treat infrastructure and application changes as the same class of production change.

Detailed Core Concepts

What DevOps is (and is not)

DevOps at TechLearningPro means product engineers and platform engineers share ownership of change safety and production outcomes. It is not “ops writes tickets while dev throws builds over a wall.” It is not “we bought a pipeline product so we are DevOps now.” Tools amplify habits; they do not replace them.

Three pillars show up in every healthy program:

1. Culture: blameless learning, shared goals (learner success and platform reliability), and visible ownership of services.
2. Practices: small batches, frequent integration, automated quality gates, environment parity, documented runbooks.
3. Automation: repeatable build, test, deploy, and observe paths that do not depend on one person’s laptop.

Without culture, automation becomes a faster way to ship bad changes. Without automation, culture becomes meetings about deploying “when we find time.” Without practices, both collapse into tribal knowledge.

The delivery loop: build → test → deploy → observe

Think of delivery as a closed loop, not a one-way conveyor:

1. Build. Compile, package, lint, produce an immutable artifact (container image digest, Lambda zip hash, CloudFormation template package). Source of truth is a Git commit SHA.
2. Test. Unit, contract, integration, security scanning, and policy-as-code checks that fail the pipeline when they fail—not “optional warnings nobody reads.”
3. Deploy. Promote the same artifact through environments (dev → staging → prod) with gates and approvals appropriate to risk. Do not rebuild for prod with different flags that were never tested.
4. Observe. Metrics, logs, traces, alarms, and user-impact signals confirm the change. Failures feed the backlog and, when needed, automatic or manual rollback.

TechLearningPro anti-pattern: build in CI, “test” manually in a shared staging that nobody owns, deploy by copying files in the console, observe only when learners tweet. That is not a loop; it is a hope.

Continuous integration, delivery, and deployment

Most TechLearningPro teams should aim for continuous delivery first: releasable mainline with a human gate on production for high-risk paths (payments, IAM, networking). Continuous deployment is a maturity choice, not a moral victory. Shipping every broken commit to prod because “we are continuous” is not DevOps.

Trunk-based development habits (without dogma)

Trunk-based development keeps the integration branch short-lived and frequent. Feature flags and progressive delivery reduce the need for long-lived release branches. You will deepen Git mechanics in lesson 25. Here, the AWS delivery point is: long-lived divergent branches force rebuild-and-hope merges that break environment promotion guarantees.

If a team still uses release branches, bind them to tagged artifacts and clear promotion rules—do not invent a second unofficial “prod branch” that only one engineer can merge.

Feedback loops and the value stream

A value stream maps every step from idea to learner value and every wait between steps. Typical TechLearningPro enrollment API change:

• Idea / ticket → design ADR (if needed) → PR → CI → staging deploy → exploratory/QA → prod approval → prod deploy → observe → learn.

Waits that kill throughput: waiting for a shared staging reset, waiting for a person with console admin rights, waiting for a manual spreadsheet of “what changed,” waiting for logs nobody can find. Automation removes waits that do not add safety. Approvals that do add safety stay—but they must be timed, owned, and auditable.

Environments and promotion

Environments are not just names. On AWS they are often:

• Separate accounts (strong isolation) or separate VPCs/stacks in one account (weaker isolation).
• Separate IAM boundaries for deploy roles.
• Separate data (never point staging at production learner PII without a deliberate, reviewed design).

Promotion means moving a proven artifact forward. Rebuilding from main “for prod” after staging tested a different commit is a classic failure mode. Tag images with digests; record the Git SHA in deployment metadata; make CloudWatch dashboards filterable by version.

Deployment strategy awareness

Lesson 34 covers blue/green, canary, and rolling in depth. For architecture today:

• Rolling: replace instances/tasks gradually; simpler; rollback means rolling forward to previous artifact.
• Blue/green: two environments; shift traffic; fast rollback by shifting back.
• Canary: small percentage of traffic sees the new version; requires good metrics.

Choose based on risk and observability readiness—not fashion. A canary without error-rate signals is theater.

AWS delivery topology (conceptual)

A practical TechLearningPro topology:

1. Source: GitHub or CodeCommit repositories for apps and infrastructure (often separate repos with clear ownership).
2. CI runners: CodeBuild projects or GitHub-hosted/self-hosted runners.
3. Artifact stores: Amazon ECR for images; S3 for pipeline artifacts and packaged templates; optionally CodeArtifact for language packages.
4. Orchestration: CodePipeline or GitHub Actions workflows with environment protections.
5. Deploy targets: ECS/Fargate, EKS, Lambda, CloudFormation/CDK/Terraform-managed stacks (later lessons).
6. Identity: IAM roles assumed by pipelines—prefer OIDC federation for GitHub (lesson 28); never long-lived access keys in secrets if you can avoid them.
7. Observe: CloudWatch, CloudTrail for who deployed what, later X-Ray/OpenTelemetry modules.

Multi-account pattern: pipeline in a shared “tooling” account deploys into workload accounts via role assumption. Detail grows in Organizations and cross-account lessons; the principle starts now—blast radius and credentials do not live in the same place as production data if you can help it.

Automation vs human gates

Automate: compile, unit tests, lint, image build, vulnerability scan, template validation, non-prod deploys, smoke tests, tagging, ticket comments with build links.

Keep humans for: production promotion of high-risk changes, architecture ADRs, break-glass emergencies (followed by retroactive PRs), security exception reviews.

Never use humans for: copying passwords into env vars by hand every release, “remembering” to run migrations, or being the only person who knows the deploy order.

Security and delivery are the same conversation

A pipeline that can deploy to prod is a privileged identity. Treat it like a production admin:

• Least-privilege deploy roles scoped to services and resources needed.
• Separate roles for build (read source, write artifacts) vs deploy (update services).
• Encrypt artifact buckets; block public access.
• Audit with CloudTrail; alert on console deploys that bypass the pipeline if policy requires pipeline-only changes.
• Secrets in Secrets Manager / Parameter Store—not in build logs, not in plaintext workflow files.

Cost and cleanup without fake prices

Pipelines cost through build minutes, artifact storage, and leftover test environments. Labs: set budget alerts; expire old artifacts with lifecycle rules; delete unused CodeBuild projects and empty buckets when the lab ends; tag Project=tlp-lesson24. Do not leave a “temporary” staging stack running for months.

Relating culture metrics to AWS signals

Useful delivery metrics (DORA-inspired, used carefully):

• Lead time for changes: commit to prod.
• Deployment frequency: how often you ship safely.
• Change failure rate: fraction of deploys that cause incidents or rollbacks.
• Time to restore: how fast you recover.

AWS signals that feed them: pipeline success/failure, deploy timestamps, alarm firings after deploy, rollback events. Do not game metrics by hiding failed deploys as “config tweaks” outside the pipeline.

Architecture and Design Diagrams

TechLearningPro delivery loop:

mermaidCopy
flowchart LR
  Idea[Idea / PR]
  Build[Build artifact]
  Test[Automated tests and gates]
  Dev[Deploy dev]
  Stage[Deploy staging]
  Gate{Prod gate}
  Prod[Deploy production]
  Obs[Observe metrics logs traces]
  Learn[Learn and backlog]

  Idea --> Build --> Test --> Dev --> Stage --> Gate
  Gate -->|approved| Prod --> Obs --> Learn --> Idea
  Gate -->|rejected| Learn
  Obs -->|rollback| Prod

Multi-environment AWS topology (conceptual):

mermaidCopy
flowchart TB
  subgraph source [Source control]
    AppRepo[App repo]
    InfraRepo[Infra repo]
  end

  subgraph ci [CI / CD]
    Runner[CodeBuild or Actions]
    Arts[ECR and S3 artifacts]
    Pipe[Pipeline orchestration]
  end

  subgraph accounts [Workload environments]
    DevAcc[Dev account or env]
    StgAcc[Staging]
    PrdAcc[Production]
  end

  subgraph obs [Observe]
    CW[CloudWatch]
    CT[CloudTrail]
  end

  AppRepo --> Runner
  InfraRepo --> Runner
  Runner --> Arts
  Arts --> Pipe
  Pipe --> DevAcc --> StgAcc --> PrdAcc
  DevAcc --> CW
  StgAcc --> CW
  PrdAcc --> CW
  Pipe --> CT

Value stream with waits called out:

mermaidCopy
flowchart TB
  A[Commit opened] --> B[CI ~ minutes]
  B --> C{Gates pass?}
  C -->|no| D[Fix on branch]
  D --> A
  C -->|yes| E[Staging deploy]
  E --> F[Smoke / QA]
  F --> G{Prod approval}
  G --> H[Prod deploy]
  H --> I[Watch burn-in window]
  I --> J[Done or rollback]

Technical Explanation

Why “build once, promote many”

If staging runs image digest sha256:abc… built from commit c1, production must run that digest—not a newly built image from c1 with a different base layer pulled later that day. Rebuilds introduce drift: dependency resolution changes, “latest” tags move, build environment updates. Immutable artifacts make promotion honest.

Implementation habits:

• Pin base images by digest in Dockerfiles when stability matters.
• Tag ECR images with Git SHA and build ID; prefer digest references in task definitions.
• Store CloudFormation/Terraform plan outputs or packaged templates as versioned artifacts.
• Record deployment metadata: who approved, which pipeline execution, which SHA.

Quality gates as architecture

Gates are not “extra CI steps.” They are architectural control points:

• Unit/contract gates protect domain invariants (enrollment cannot double-seat).
• Integration gates protect AWS wiring (IAM, VPC, event contracts).
• Security gates protect the supply chain (dependency CVE policy, secret scanning).
• Policy gates protect platform standards (required tags, denied public S3, encryption).
• Manual approval gates protect high-blast-radius changes when automation confidence is incomplete.

Fail closed on security and policy for production paths. Fail open only with explicit, time-bounded exceptions logged in an ADR.

Environment parity

Parity means staging resembles production in shape: similar networking patterns, similar IAM model, similar data volume characteristics (not necessarily full prod data). Perfect clones are expensive; intentional differences must be documented (smaller instance sizes, synthetic data).

Dangerous non-parity: staging without auth; staging with *:* IAM; staging using public buckets “for ease.” Those train engineers to ship unsafe defaults.

Observe is part of deploy

A deploy is incomplete until:

• Health checks pass.
• Error-rate and latency signals remain within burn-in thresholds.
• Dashboards show the new version.
• On-call knows the change window.

Wire pipeline success notifications with links to the execution and the dashboard. Later modules deepen CloudWatch and tracing; Module 3 requires you to leave hooks for them.

Console and break-glass

Break-glass console access exists for outages. Rules:

1. Time-bounded elevated access.
2. Ticket or incident ID required.
3. CloudTrail reviewed.
4. Retroactive IaC/PR within an agreed SLA so Git remains source of truth.

If break-glass is weekly, your delivery architecture failed—fix the pipeline, not the people using the ladder.

Mapping Module 2 services into the loop

Pipeline identity model (preview)

• CodeBuild service role: logs, pull source, push artifacts.
• CodePipeline role: start actions, read/write artifact bucket.
• Deploy role in target account: ecs:UpdateService, cloudformation:* scoped, etc.—least privilege, not AdministratorAccess.
• GitHub Actions: OIDC → AssumeRoleWithWebIdentity with sub claims limited to repo and environment (lesson 28).

Never teach long-lived IAM user access keys as the default for CI to AWS.

Organizational interfaces

Delivery architecture includes people interfaces:

• Platform team owns shared pipeline templates, OIDC providers, artifact bucket policies.
• Product teams own app buildspecs/workflows and service-level dashboards.
• Security owns guardrails and exception process.
• FinOps owns budget alerts and cost allocation tags on pipeline and environment resources.

RACI for “who can approve prod” must be written. Ambiguity becomes either endless waits or reckless self-approval.

Practical Implementation or Design Walkthrough

Problem

TechLearningPro’s enrollment API changes sometimes ship via laptop deploys. Staging drifts from prod. Nobody can answer “what version is live?” during an incident. Leadership asks for “more DevOps.”

Steps

1. Draw the current value stream on a whiteboard: every handoff and wait for the last five enrollments-related changes.
2. Declare source of truth: Git repos for app and infra; no shadow prod.
3. Define environments: at least sandbox/dev, staging, prod—with account or stack boundaries and deploy roles.
4. Pick artifact types: container digest for the API; packaged template for infra.
5. Specify gates: unit + lint on PR; integration on trunk; security scan; staging smoke; prod approval for payment-adjacent changes.
6. Specify observe hooks: alarm on 5xx rate and p95 latency; dashboard panel for “deployments.”
7. Write an ADR capturing continuous delivery (not yet continuous deployment), OIDC preference, and break-glass rules.
8. Set budget alert before creating any shared artifact bucket in the sandbox.
9. Run a dry-run: one trivial change (health endpoint text) through the loop end-to-end.
10. Cleanup plan: lifecycle rules and deletion checklist for lab resources.

Failure paths to rehearse

• CI green but staging smoke fails → block promotion; fix forward on trunk.
• Prod deploy succeeds but error rate climbs → rollback to previous digest; open incident.
• Engineer console-edits a security group → detect via CloudTrail; revert via IaC; retrospective.

Deliverable sketch (value stream map)

Document for each step: owner, tool, SLA, evidence location (pipeline URL, CloudWatch alarm ARN). Submit with the lab evidence checklist in Hands-On Exercises.

Minimal “pipeline blueprint” narrative

Even before you write YAML in later lessons, write this blueprint:

textCopy
trigger: pull_request and push to main
build: language toolchain + docker build
test: unit, lint, contract
publish: push image digest to ECR (non-prod repo or tag namespace)
deploy_dev: update service with digest
deploy_staging: same digest + smoke
approve_prod: named environment gate
deploy_prod: same digest
observe: burn-in 30–60 minutes with rollback runbook link

ADR template (short)

• Context: laptop deploys and staging drift.
• Decision: continuous delivery with immutable artifacts; prod approval gate; OIDC for GitHub; separate deploy roles.
• Consequences: slower first setup; faster safe shipping; console changes become exceptions.

Enterprise Real-World Scenarios

“We installed a pipeline; culture unchanged”

A team automates build but still requires a weekly CAB for every CSS change and still hotfixes prod from laptops. Symptom: pipeline adoption metrics look good; change failure rate unchanged. Fix: shrink batch size, redefine CAB for true risk, ban routine laptop prod deploys.

Shared staging owned by nobody

Five squads overwrite each other’s stacks. Tests flake. People bypass staging. Fix: per-squad ephemeral environments or strict scheduling plus infrastructure ownership; prefer account-per-env when noise is chronic.

Continuous deployment without observability

Every merge goes to prod; dashboards are empty; on-call learns from Twitter. Fix: stop continuous deployment until burn-in signals exist; keep continuous delivery with a gate.

AdminAccess on the pipeline role

Build role can create IAM users and open security groups. A supply-chain compromise becomes account takeover. Fix: split roles; scope actions; add permission boundaries where Organizations policy requires them.

Rebuilding for production

Staging tested Monday’s build; Friday’s “same code” rebuild picks a new base image with a critical bug. Fix: promote digests; pin bases; record SBOM where your supply-chain lesson requires it.

Metrics theater

Team maximizes deployment frequency by splitting no-op deploys. Learners still see weekly outages. Fix: pair frequency with change failure rate and customer SLIs—not vanity counters.

Best Practices

1. Treat Git as the source of truth for app and infrastructure.
2. Build once; promote the same artifact across environments.
3. Prefer small batches and short-lived branches.
4. Automate every gate that is objective; keep humans for judgment and high risk.
5. Separate build and deploy IAM roles; avoid long-lived access keys for CI.
6. Encrypt and lock down artifact storage; block public access.
7. Require staging smoke tests before prod gates.
8. Attach observe and rollback to every production deploy definition.
9. Document break-glass and enforce retroactive PRs.
10. Tag delivery resources for cost allocation; set budgets.
11. Measure lead time and change failure rate honestly.
12. Align platform and product ownership with clear RACI.
13. Cross-link pipeline runs to commits, tickets, and dashboards.
14. Design for audit: who approved prod, which digest shipped.
15. Cleanup lab pipelines and buckets; do not orphan spend.

Common Mistakes and Anti-Patterns

1. DevOps as a silo team that only runs tools — Symptom: product teams still throw builds. Cause: org chart theater. Fix: shared ownership and self-service pipelines.
2. Latest tags in production — Symptom: mystery versions. Cause: mutable tags. Fix: digests and immutable tags.
3. Manual prod checklist instead of pipeline — Symptom: missed steps. Cause: fear of automation. Fix: encode the checklist as gates.
4. No budget on CI accounts — Symptom: surprise bills from stuck builds or retained artifacts. Cause: missing FinOps. Fix: budgets + lifecycle policies.
5. Secrets in plaintext workflow env — Symptom: leaked credentials. Cause: convenience. Fix: OIDC + Secrets Manager; rotate anything that leaked.
6. Skipping staging because “it’s urgent” — Symptom: urgent becomes outage. Cause: process debt. Fix: fast path still uses staging smoke; improve pipeline speed.
7. One mega-pipeline for all services — Symptom: queueing and fragile blasts. Cause: centralization fetish. Fix: templates + per-service pipelines.
8. Ignoring CloudTrail on deploy roles — Symptom: unknown changes. Cause: observe gap. Fix: alarms on unusual AssumeRole and console deploys.

Troubleshooting and Design Review

Pipeline is green but prod is wrong version. Check whether deploy referenced a floating tag; compare running digest to artifact store; verify promotion used the same execution artifacts.

Lead time is days for one-line fixes. Value-stream the waits: review SLA, flaky tests, shared env locks, approval offline. Fix the largest wait first.

Staging never matches prod. Diff IAM, networking, feature flags, and data shape. Document intentional differences; eliminate accidental ones.

On-call cannot find what changed. Require deployment events in the same dashboard as SLIs; include Git SHA in service telemetry resource attributes.

Design review checklist: Is the loop closed (observe → learn)? Are artifacts immutable? Are roles least-privilege? Is break-glass defined? Is budget set? Is prod gate risk-based rather than political?

CLI orientation (verify flags in current AWS CLI docs): list recent CodePipeline executions; describe ECS service task definition digests; get CloudTrail events for AssumeRole—use as investigation starting points, not as memorized trivia.

Interview Questions and Answers

Q1: What is DevOps?
A: Culture, practices, and automation that shrink the path from idea to safe production change and from production signal back to improvement—not merely a toolchain.

Q2: CI vs continuous delivery vs continuous deployment?
A: CI integrates and tests frequently; delivery keeps mainline releasable with a deliberate prod decision; deployment releases every passing change automatically.

Q3: Why build once and promote many?
A: Rebuilds introduce drift; immutable artifacts ensure staging evidence applies to production.

Q4: Where do humans belong in AWS delivery?
A: Architecture decisions, high-risk approvals, break-glass—with audit—not routine button clicking that should be automated.

Q5: How should GitHub authenticate to AWS?
A: Prefer OIDC federation assuming IAM roles with constrained claims—not long-lived access keys.

Q6: What is a value stream map used for?
A: Finding waits and handoffs that dominate lead time so you automate or redesign them.

Q7: Why separate build and deploy roles?
A: Least privilege and blast-radius reduction if the build system is compromised.

Q8: When is continuous deployment a bad idea?
A: When observability, test confidence, or rollback paths are weak—especially for payment and IAM changes.

Q9: How do you detect laptop prod deploys?
A: CloudTrail and config rules / policy; cultural enforcement via retroactive IaC requirements.

Q10: What metrics would you show leadership?
A: Lead time, deploy frequency, change failure rate, time to restore—tied to learner-facing SLIs, not vanity.

Q11: How do Module 2 services fit Module 3?
A: They are deploy targets and contracts; Module 3 versions, reviews, and ships them safely.

Q12: What belongs in an ADR for delivery?
A: Promotion model, auth to AWS, environment boundaries, gate policy, break-glass, and rollback expectations.

Hands-On Exercises

Exercise: Map a delivery value stream and identify automation controls

Goal: Produce a one-page value stream for TechLearningPro’s enrollment API change path and mark each step as automate / human gate / eliminate.

Steps:

1. Pick a recent real or hypothetical change (health text, feature flag, pricing rule).
2. List steps from idea to observe.
3. Time-box estimated waits (even rough orders of magnitude).
4. Mark AWS controls you would add (CodeBuild gate, OIDC role, ECR digest, CloudWatch burn-in).
5. Set or verify a budget alert in the sandbox account before any resource creation in follow-on labs.
6. Write cleanup notes for any resource you would create (bucket lifecycle, pipeline deletion).

Evidence: Diagram (Mermaid or photo of whiteboard), ADR stub, budget alert screenshot or CLI describe output, checklist of automate vs human.

Challenge ADR

Extend the design with a production constraint: multi-account isolation for prod vs non-prod, monthly budget ceiling for CI minutes, and a requirement that payment-service deploys need dual approval. Document trade-offs: slower prod vs lower blast radius.

Lab evidence checklist

• Value stream with waits labeled
• Artifact strategy (image digest / template package)
• Gate list for PR, trunk, staging, prod
• Observe hooks named
• Break-glass rule written
• Budget alert confirmed
• Cleanup plan written
• Links to next lessons for implementation

Narrative: enrollment week war room

Imagine Monday 09:00 IST. Error rates climb after a “quick” console config change. Using this lesson’s topology, narrate how you identify the change (CloudTrail), roll back (previous digest), and file the follow-up (pipeline gate + IaC). Practice the story until it is calm and specific.

Connecting Git and CI primers

Skim /learn/git only for commit/review hygiene vocabulary and /learn/ci-cd-automation for generic stage language. Then return here: your exams and projects will grade AWS-shaped answers—roles, artifacts, environments—not generic slogans.

Deep dive: DORA metrics without cargo cult

Write a paragraph on how TechLearningPro would instrument lead time using pipeline timestamps and Git SHAs. Note pitfalls: excluding failed deploys, measuring only “happy” services, or punishing teams for incident honesty.

Environment matrix worksheet

Create a table: Environment | Account | Deploy role | Data class | Approval | Observe dashboard. Fill for sandbox, staging, prod. Leave blanks only where an ADR explicitly postpones a decision.

Anti-pattern clinic: mutable “prod” tag

Describe how a floating prod tag on ECR causes a rollback to deploy the wrong image. Propose tag + digest policy for lesson 19’s supply-chain habits applied to delivery.

Platform vs product RACI mini-table

Rows: OIDC provider, pipeline template, service buildspec, prod approval, alarm threshold, budget. Columns: Platform, Product, Security. Mark R/A/C/I. Ambiguity is a finding.

Observability burn-in window design

Choose 15 vs 60 minutes for enrollment API burn-in. Justify with traffic patterns (class-start spikes) and rollback speed. Document who watches and what auto-rollback would require later.

Security review script for delivery

Checklist: no long-lived keys; artifact bucket public access blocked; deploy role lacks iam:CreateUser; secrets not echoed in logs; CloudTrail enabled; prod gate cannot be skipped by renaming branches.

Cost control narrative for leadership

Explain in six sentences why artifact lifecycle rules and ephemeral preview environments reduce cost and increase safety. Avoid inventing currency amounts; speak in drivers (storage, build minutes, idle environments).

Failure injection catalog

List five delivery failures to rehearse: failed unit gate, failed staging smoke, approval timeout, deploy API throttling, observe alarm during burn-in. For each: expected pipeline behavior and human action.

Relationship to Project 2

Project 2: Container CI/CD Platform will demand this topology in concrete pipelines. Your value stream and ADR become the project’s starting architecture brief—keep them.

Preview bridge to lesson 25–29

Lesson 25 governs the repos; 26 formalizes pipeline semantics; 27 and 28 implement on AWS/GitHub; 29 begins IaC with CloudFormation so infra rides the same loop. Do not skip governance because “we will just script it.”

Writing the “definition of done” for a deploy

Draft DoD: digest matches artifact; smoke green; alarms quiet; dashboard updated; ticket commented; rollback command known. Hang it in the team wiki via a PR—yes, even the wiki link belongs in Git if it is a standard.

Multi-region awareness (light)

Note that active-active multi-region delivery doubles promotion complexity. For this lesson, document whether TechLearningPro enrollment is single-region active or warm standby—and refuse silent partial regional deploys.

Human gate SLA

Set an approval SLA (for example 30 minutes during business hours) so continuous delivery does not become continuous waiting. Offline approvals need deputies.

Mapping Well-Architected operational excellence

Translate this lesson’s loop into Well-Architected operational excellence practices: evolve procedures, anticipate failure, learn from ops events. Cite your foundations lesson rather than inventing new pillars.

Student privacy and staging data

State a hard rule: no production learner PII in personal sandboxes. Synthetic data or anonymized subsets only with legal/platform approval. Delivery architecture includes data ethics.

Pipeline as a product

Treat shared pipeline templates like internal products: version them, changelog them, avoid breaking consumers. Product teams should not fork templates without a plan to merge back.

Game day: laptop ban drill

For one week in a lab org, disallow routine console prod changes. Measure how many true emergencies remain. Use the result to prioritize pipeline gaps.

Cross-link recap for learners

You will implement with CI/CD Pipeline Fundamentals, AWS CodeBuild and CodePipeline, and GitHub Actions for AWS Deployments. Infrastructure joins via Infrastructure as Code with CloudFormation and later Terraform/CDK. Module 2 targets remain the things you ship.

What “safe to proceed” means after observe

Define quantitative thresholds for the enrollment API (error rate, latency, saturation). Without numbers, observe is vibes. Wire those numbers into the burn-in checklist even if auto-rollback comes later.

Change failure taxonomy

Classify failures: bad artifact, bad config, bad approval decision, bad observe (false calm), bad rollback. Each class needs a different fix in the delivery architecture.

Documentation minimum viable set

README for local run; CONTRIBUTING for PR gates; OPERATIONS for rollback; ADR folder for delivery decisions. All live in Git—/learn/git discipline applied to ops docs.

Closing rehearsal questions

Answer aloud: What is our artifact? What is our prod gate? What is our rollback unit? What is our budget? If any answer is vague, the lesson is not done.

Expanded walkthrough: one feature through the loop

Consider a TechLearningPro change that adds a Retry-After header on enrollment rate limits. An engineer opens a short-lived branch, writes unit tests for the header behavior, and opens a pull request. CODEOWNERS (lesson 25) routes review to the API guild. CI builds the container, runs unit and contract tests against a stubbed seat service, and publishes an image tagged with the commit SHA to the non-production ECR namespace. After merge to trunk, the pipeline deploys that digest to the development environment automatically. Staging receives the same digest after integration tests against a shared EventBridge contract suite pass. A smoke test enrolls a synthetic learner and asserts the header appears under forced 429s. Production waits on an environment approval because rate-limit behavior can amplify support load if wrong. After approval, production ECS service updates to the digest. On-call watches p95 latency and 5xx for thirty minutes. Support receives a release note with the SHA. The loop closes when a dashboard annotation marks the deploy time so later incidents can correlate. Nothing in this story required a laptop with AdministratorAccess. Everything produced evidence: PR, pipeline execution, digest, approval record, burn-in notes.

Capacity and concurrency of delivery

Delivery systems themselves have capacity. If every microservice rebuilds the world on each commit with no caching, CodeBuild queues explode during enrollment week freezes. Design build caches thoughtfully; split slow integration suites to post-merge where policy allows; keep PR feedback under a team SLA (for example ten minutes for unit+lint). Capacity planning for CI is part of delivery architecture—ignore it and engineers will bypass CI “just this once.”

Compliance narrative without bureaucracy theater

Some TechLearningPro enterprise customers ask who approved production changes affecting their tenant data path. Your topology should answer with pipeline execution history, IAM principals, and Git SHAs—not with a slide deck assembled after the fact. That is compliance as a byproduct of good delivery, not a parallel paperwork universe. When auditors ask for evidence, point at systems of record.

Progressive delivery preview

Feature flags and weighted traffic shifts let you separate deploy (binary present in prod) from release (learners see behavior). Module 3 later covers deployment strategies; the principle belongs in today’s architecture: the loop can deploy dark, observe, then release. Do not conflate “merged to main” with “visible to all learners.”

Local development vs delivery truth

Local Docker Compose helps developers move fast; it is not an environment in the promotion chain. Never claim “works on my machine” as staging evidence. Contract tests and ephemeral cloud environments exist to close that honesty gap. Document which checks are local-only and which are release gates.

Incident learning re-entering the backlog

After a failed deploy, the retrospective action items must become backlog items with owners—new gates, better alarms, clearer runbooks. If learnings die in a doc nobody reads, observe does not connect to build. Require at least one pipeline or alarm improvement from each Sev-1 related to change failure.

Tagging standard for delivery resources

Propose tags: Project, Environment, Service, Owner, CostCenter, Pipeline. Enforce via policy-as-code later (lesson 35). Without tags, FinOps cannot attribute build spend to squads, and cleanup scripts cannot find orphans safely.

When not to automate yet

If a process is wrong, automating it ships wrongness faster. Example: a manual twenty-step prod checklist that includes contradictory steps. First simplify the checklist to a correct minimal set, then automate. Delivery architecture includes the courage to delete ceremony that does not reduce risk.

Dual-tracked app and infra changes

Sometimes an API needs a new IAM permission and a code change. Teach engineers to sequence safely: infra permission lands first (or in the same coordinated pipeline), then app. Or use feature flags so app tolerates missing permission until infra arrives. Uncoordinated dual track is a top cause of “works in staging” surprises when staging roles are looser than prod.

Hands-on peer review of another squad’s map

Exchange value stream maps with a peer. Score them: artifact immutability clarity, gate objectivity, observe hooks, break-glass honesty, budget mention. Peer pressure here is pedagogical, not punitive—vague maps fail the rubric the same way vague IAM policies fail production.

Explicit non-goals for this lesson

This lesson does not teach buildspec syntax, Actions YAML, or CloudFormation intrinsic functions—that is deliberate. Jumping to YAML before topology produces cargo-cult pipelines. You will write those artifacts in lessons 26–29 with this mental model as the acceptance criteria.

Oral exam style prompts

Explain to a staff engineer: why OIDC over keys; why digest over latest; why continuous delivery before continuous deployment for payments; why CloudTrail matters to DevOps culture. If answers need slides, they are not yet internalized.

Worked example: measuring lead time for one change

Start clock at first commit timestamp on the feature branch. End clock at production pipeline success time for the digest that contains that commit. Subtract waiting on human review and waiting on prod approval as separate annotated segments so you can improve each. If lead time is forty-eight hours but coding was two hours, the architecture problem is waits—not engineer skill. Bring that graph to the next platform sync.

Worked example: change failure rate honesty

Count production deploys over a month. Count those that caused a rollback, a Sev-1/2, or a hotfix within twenty-four hours. Divide. Do not exclude “config-only” deploys if they changed learner traffic. Do not hide failures by undeploying quietly without an incident ticket. Honest rates fund better gates; dishonest rates fund surprise outages.

Sandbox budget drill

In the AWS Billing console or via AWS Budgets, ensure an alert exists for the sandbox account before Module 3 labs create ECR repos, S3 artifact buckets, or CodeBuild projects. Record the threshold you chose and why. Cleanup is part of every later exercise; today’s job is to make surprise spend unlikely.

Executive one-pager exercise

Write a one-page memo to TechLearningPro leadership: problem (laptop deploys), proposal (delivery loop on AWS), ask (time for platform templates and OIDC), risk (slower first month), return (faster safe lead time, auditability). No buzzword salad. This memo is a career skill as much as a lab artifact.

Final integration checklist before lesson 25

Confirm you can explain the loop without notes; confirm you know where Git, artifacts, environments, gates, and observe sit; confirm you will not paste long-lived AWS keys into any workflow as you proceed. Then move to repository governance with intent.

Rubric self-score before submission

Score yourself 1–5 on: conceptual accuracy of the loop; realism of the value stream; security of the identity model; observability hooks; documentation clarity. Anything below 4 becomes tonight’s study target. The evaluation rubric in the brief weights working reproducibility and security heavily—your diagram alone is not enough without gates, budget, and cleanup thinking.

From Module 2 scars to Module 3 habits

If you ever hotfixed a Lambda in the console during Module 2 labs, name that behavior explicitly in your value stream as a risk. Module 3 exists so that behavior becomes rare, audited, and replaced by pipeline promotions. Shame is useless; replacing the path is useful. Carry one personal “I will not” rule into lesson 25’s branch protections.

Glossary pocket card

Artifact: immutable build output promoted unchanged. Gate: automated or human check that can stop promotion. Environment: isolated deploy target with its own identity and data class. Burn-in: post-deploy observe window before declaring success. Break-glass: emergency console path with audit and retroactive PR. Keep this card visible while you draft the value stream so vocabulary stays precise under time pressure. Print it or pin it beside your ADR stub until the language becomes automatic in design reviews.

Lesson Summary

DevOps at TechLearningPro is a closed loop—build, test, deploy, observe—backed by culture, small batches, and automation that respects blast radius. Prefer continuous delivery with immutable artifacts, least-privilege pipeline roles, risk-based human gates, and honest metrics. AWS delivery topology spans Git, CI runners, ECR/S3 artifacts, environment promotion, and observability, with OIDC preferred over long-lived keys.

Next: Git Workflows and Repository Governance makes the source side of this architecture enforceable with CODEOWNERS, protected branches, and review rules for both app and infrastructure repositories.

Further Reading

• AWS Well-Architected Framework — Operational Excellence
• Practicing Continuous Integration and Continuous Delivery on AWS (AWS Whitepaper)
• AWS CodePipeline User Guide
• AWS CodeBuild User Guide
• Amazon ECR User Guide
• IAM roles for cross-account access
• AWS CloudTrail User Guide
• Tagging AWS resources
• AWS Budgets
• DevOps Research and Assessment (DORA) metrics overview (Google Cloud architecture docs)