AWS Cloud & DevOps Tutorial 0/70 lessons ~6 min read Lesson 62“IAM, VPC, EC2, Lambda, ECS, EKS, Terraform, CI/CD, and AWS operations”

Kubernetes Operations and GitOps on EKS

Lesson 18 introduced Amazon EKS without cloning [/learn/kubernetes](/learn/kubernetes). Production TechLearningPro services on EKS need **operations**: probes, requests/limits, Pod

Course progress0%

Focus

15 guided sections

Practice signal

Examples included

Career prep

Interview Q&A included

Kubernetes Operations and GitOps on EKS

AWS Cloud & DevOps Engineering · Lesson 62 · Production Projects and Interview

Lesson Overview

Lesson 18 introduced Amazon EKS without cloning /learn/kubernetes. Production TechLearningPro services on EKS need operations: probes, requests/limits, PodDisruptionBudgets, Cluster/Node autoscaling concepts, managed node group hygiene, safe upgrades, Helm as packaging, and GitOps (Argo CD or Flux concepts) so desired state lives in git—not laptop kubectl apply. This lesson focuses on operating and evolving an EKS workload path for progress/enrollment APIs: digest-pinned images from lesson 61, drift detection, rollback via git revert, and budget-aware clusters that get deleted after labs.

Deep API object theory stays in /learn/kubernetes; image build depth in /learn/docker. Tag Project=tlp-lesson62; EKS and node groups are expensive—budgets and cleanup are part of the grade.

Learning Objectives

Operate Deployments with liveness/readiness/startup probes appropriate to progress API.
Set CPU/memory requests and limits deliberately; explain noisy-neighbor risk.
Use PodDisruptionBudgets to protect capacity during drains/upgrades.
Explain Cluster Autoscaler or Karpenter concepts at a high level without inventing flags.
Manage managed node groups: AMI updates, scaling, taints/labels basics.
Plan EKS control plane and node upgrade order using current AWS guidance.
Compare Helm packaging versus raw manifests for TechLearningPro services.
Explain GitOps reconciliation (Argo CD/Flux concepts): desired state in git.
Demonstrate rollback and drift correction via git, not snowflake kubectl.
Pin image digests in manifests; refuse latest.
Wire IRSA concepts for AWS API access from pods.
Observe with CloudWatch Container Insights / Prometheus options at concept level.
Cost-control: scale to zero non-prod, delete lab clusters.
Cross-link Kubernetes course for RBAC/controllers depth.
Prepare interview answers on GitOps vs CI push.
Document PDB + probe choices in an ADR.
Negative test: broken probe → service isolation behavior.
Negative test: git drift corrected by reconciler.
Security: least privilege RBAC for deployers vs humans.
Hand off evidence into PRR lesson 63.

Prerequisites

Amazon EKS and Kubernetes on AWS.
Secure Software Supply Chain and DevSecOps.
/learn/kubernetes skim for Deployment/Service/Ingress vocabulary.
/learn/docker for image literacy.
Sandbox + strict budget; prefer small node types; delete cluster after exercise.

Detailed Core Concepts

Operations versus cluster creation

Creating EKS is not operating EKS. Operations means: workloads survive node rotation; probes match real health; resources are sized; disruptions are budgeted; upgrades are rehearsed; git is source of truth; digests are pinned; on-call has runbooks (lesson 63).

Probes

Readiness — remove from Service endpoints when not ready (dependency warm-up).
Liveness — restart container when deadlocked (careful: too aggressive kills recovering apps).
Startup — delay liveness for slow boots.

TechLearningPro progress API readiness should check critical dependency light checks—not only TCP port open if the app accepts sockets before migrations finish.

Requests, limits, and scheduling

Requests drive scheduling and autoscaling signals; limits cap bursts. CPU throttling and OOM kills are common misconfig symptoms. Align with lesson 59 host OOM literacy when using node groups.

PodDisruptionBudgets (PDBs)

During node drain/upgrade, PDBs keep minimum available pods. Without PDBs, upgrades can evacuate an entire Deployment. Define minAvailable or maxUnavailable consciously for each tier.

Node groups and upgrades

Managed node groups simplify worker AMI updates. Upgrade posture typically: read EKS release notes → upgrade control plane to supported version → upgrade nodes → upgrade add-ons (verify current AWS recommended order in docs). Test in non-prod. Respect PDBs and capacity.

Autoscaling concepts

Cluster Autoscaler / Karpenter add nodes when pods pending; HPA scales pods on metrics. Understand pending pods due to wrong requests versus genuine need. Karpenter/CA exact install flags change—learn concepts; verify docs for lab.

Helm

Helm packages Kubernetes manifests with values for env differences. Still pin chart versions and image digests. Helm is not GitOps by itself; GitOps can reconcile Helm releases.

GitOps (Argo CD / Flux concepts)

Git holds desired state; a controller reconciles cluster toward git. Benefits: PR review, drift detection, rollback=git revert. Secrets still not in git—use Sealed Secrets/ESO/ASM patterns (conceptual). CI builds/pushes images and updates manifest digest; GitOps deploys. Avoid CI with broad kubectl apply to prod if GitOps is the paved path—two sources of truth fight.

Drift

Manual hotfix via kubectl creates drift; reconciler may revert or alert depending on policy. Prefer hotfix branch + PR even under pressure, or documented break-glass with follow-up PR.

Architecture and Design Diagrams

mermaid
flowchart TB
  Git[Git desired state]
  PR[PR review]
  GC[GitOps controller]
  EKS[EKS API]
  Pods[Pods / Services]
  ECR[ECR digests]
  NG[Managed node group]

  PR --> Git --> GC --> EKS --> Pods
  ECR --> Pods
  NG --> Pods

mermaid
sequenceDiagram
  participant Dev as Developer
  participant Git as Git
  participant G as GitOps
  participant C as Cluster
  Dev->>Git: Merge digest pin
  G->>Git: Detect change
  G->>C: Reconcile Deployment
  C-->>G: Healthy

Technical Explanation

Rollback story

Bad digest merged; readiness fails; GitOps keeps trying or marks degraded (tool-dependent). Revert commit → previous digest returns. ECR lifecycle must still hold previous digest (lesson 61).

Upgrade story

Non-prod cluster upgraded first; PDB verified; node group rolling update; add-ons compatibility checked against EKS version skew policy in AWS docs; then stage; then prod change window.

IRSA reminder

Pods need IAM roles for S3/SQS/etc. via IRSA or successor pod identity mechanisms—verify current recommendation. Node instance roles should not be the app’s AWS credential source.

Cost

Control plane hourly + nodes + LB + NAT + ECR pulls. Lab: smallest viable nodes, short TTL, delete cluster. Tag Project=tlp-lesson62.

Cross-links

/learn/kubernetes for controllers/RBAC depth; /learn/docker for image builds; this lesson for AWS EKS ops + GitOps operating model.

Extended depth: GitOps operating model for TechLearningPro

CI builds and pushes digests to ECR, then opens or updates a PR that changes only the image field (digest) and chart values needed for that release. Humans review the PR for blast radius. Merge triggers reconcile. Production cluster credentials stay with the GitOps controller service account—not with every developer laptop. Break-glass kubectl is logged, time-bounded, and followed by a sync PR so git catches up.

Extended depth: upgrade rehearsal checklist

Read EKS release notes and add-on compatibility for the target version (verify current AWS docs).
Snapshot or backup notes for any stateful components you actually run on the cluster.
Ensure PDBs and spare capacity exist before draining nodes.
Upgrade non-prod control plane; validate workloads and CNI/DNS add-ons.
Roll node groups; watch pod evictions and service health.
Only then schedule staging and production windows.
Keep a rollback story: version skew limits may prevent easy downgrade—confirm in AWS docs before you need it.

Extended depth: cost kill-switch

Lab clusters tagged Project=tlp-lesson62 must have an owner and destroy-by timestamp in the README. If Cost Explorer shows unexpected NAT or node hours, destroy first, debug architecture second. Premium courses grade FinOps as part of Kubernetes operations.

Extended depth: cross-link without cloning

Pod lifecycle, controllers, and RBAC deep dives belong in /learn/kubernetes. Image layer hygiene belongs in /learn/docker. Supply-chain signing gates belong in lesson 61. This lesson is the EKS operations and GitOps control plane for TechLearningPro.

Practical Implementation or Design Walkthrough

Problem

Deploy a sample app using GitOps concepts on EKS (or design-only if budget-constrained) showing rollback and drift correction.

Steps

Budget alert; plan max spend; set calendar delete time.
Create or reuse small EKS cluster + managed node group (or document design-only with evidence of cost refusal).
Write Deployment+Service manifests with probes, requests, PDB, digest pin (public sample image ok).
Install or describe GitOps controller (Argo/Flux)—follow current project docs; no invented flags.
Commit manifests; show sync; change a label; show reconcile.
Break readiness deliberately in a branch; observe traffic isolation; revert.
Document node upgrade dry-run checklist.
ADR: GitOps vs CI kubectl apply for TechLearningPro.
Tag resources; delete cluster/node groups/VPCs as needed; evidence.
Deliver sync screenshots/logs + rollback commit SHAs.

Enterprise Real-World Scenarios

kubectl snowflakes

Drift fights GitOps nightly. Fix: disable human write or alert on drift; PR hotfixes.

No PDBs

Upgrade evacuated progress API. Fix: PDBs + surge capacity.

latest tag

Silent bad pull. Fix: digests + policy.

Cluster left on

FinOps spike. Fix: TTL, budgets, delete checklist.

Control plane only upgrade

Addon skew breaks CNI. Fix: follow AWS version skew guidance.

Best Practices

Git as desired state for cluster apps.
Digest pins only.
Probes match real readiness.
PDBs on tier-1 services.
Rehearse upgrades in non-prod.
Separate build CI from GitOps deploy.
IRSA for AWS access.
Budget and delete lab clusters.
Pin Helm chart versions.
Document break-glass kubectl.
Observe pending pods vs node pressure.
Tag node groups and cluster.
Cross-train on /learn/kubernetes.
Rollback via git revert.
Feed evidence to PRR.

Common Mistakes and Anti-Patterns

kubectl apply as only prod path while claiming GitOps.
No resource requests.
Liveness on dependencies that flake.
Skipping PDB on upgrades.
Leaving EKS lab running all month.
Floating latest images.
Node IAM role used by apps.
Inventing eksctl flags from memory.
Ignoring release notes/skew.
Secrets committed to git.

Troubleshooting and Design Review

Symptom	Cause	Action
Pods pending	Requests/taints/capacity	Describe pod events; scale nodes or fix requests
CrashLoop	Bad digest/config	Logs; revert git
GitOps out of sync	Manual drift or failed apply	Diff; fix RBAC; reconcile
Upgrade stuck	PDB/capacity	Add capacity; review PDB
ImagePullBackOff	IRSA/ECR perms/digest missing	Check pull role and digest existence
Cost alarm	Idle nodes	Scale down/delete; review NAT

Interview Questions and Answers

Q1: GitOps benefit? A: PR-reviewed desired state + drift control. Q2: Readiness vs liveness? A: Traffic vs restart. Q3: PDB purpose? A: Protect availability during voluntary disruption. Q4: Digest pin? A: Immutable bytes from supply chain. Q5: Upgrade order? A: Follow current AWS guidance; non-prod first. Q6: Helm role? A: Package/templates; still needs GitOps/pinning. Q7: HPA vs cluster autoscaler? A: Pods vs nodes. Q8: Drift? A: Live state ≠ git; reconciler corrects/alerts. Q9: IRSA? A: Pod AWS creds without node role sharing. Q10: Rollback? A: Git revert to prior digest. Q11: Why not only CI apply? A: Dual truth; weaker drift story. Q12: Cost control? A: Small labs, delete clusters, rightsize. Q13: Link K8s course? A: API depth there; EKS ops here. Q14: Probe anti-pattern? A: Liveness hitting dependent DB hard. Q15: Next? A: PRR and runbooks consuming this evidence.

Hands-On Exercises

Exercise: GitOps deploy with rollback and drift correction

Deliverable: manifests with probes/PDB/digest, GitOps sync evidence, revert commit, upgrade checklist, Project=tlp-lesson62 destroy proof (or design-only cost waiver note).

Challenge ADR

Adopt Flux or Argo CD as sole prod deploy path in 60 days; CI loses cluster admin.

Lab evidence checklist

Cluster ARN; sync log; rollback SHA; budget screenshot; delete confirmation.

Narrative: PDB saves the upgrade

Node group rolling update without PDB blacked out enrollment; after PDB+extra capacity, upgrades became boring—the goal.

Continuity

Next: Production Readiness Reviews and Runbooks. Deepen at /learn/kubernetes.

Extended practice scenario 1 — 62 kubernetes operations and gitops on eks

Confirm AWS Budgets alerts exist for the sandbox; tag every resource Project=tlp-lesson62. Do not proceed without a budget subscriber.

In a timed drill for Kubernetes operations and GitOps on EKS at TechLearningPro, implement or design the core lab with a partner observing. The observer's job is to stop any step that invents API flags, hardcodes secrets, invents pricing numbers presented as current quotes, or skips budget and cleanup. After the drill, rewrite your ADR in ten lines: context, decision, consequences. Attach evidence: manifests, sync logs, rollback SHA, upgrade checklist, destroy proof, and destroy confirmation for any lab resources.

Discuss how Kubernetes operations and GitOps on EKS connects prior AWS lessons (IAM, VPC, EC2/SSM, CI/CD, observability, FinOps) and upcoming Module 6 projects. List two EKS ops risks before Project 2, with owners and dates.

Finally, teach back the lesson overview paragraph from memory, then check against the text. Gaps become flashcards. Premium courses finish when you can operate under time pressure without treating Kubernetes operations and GitOps on EKS as a mystery.

Extended practice scenario 2 — 62 kubernetes operations and gitops on eks

Document the lesson's primary architecture decision in a ten-line ADR: context, decision, status, consequences.

Extended practice scenario 3 — 62 kubernetes operations and gitops on eks

Map identity paths: who can create/modify the resources in this lesson; which IAM roles are least privilege.

Extended practice scenario 4 — 62 kubernetes operations and gitops on eks

Design observability: CloudWatch metrics/alarms or equivalent signals that prove the control works.

Extended practice scenario 5 — 62 kubernetes operations and gitops on eks

Run or design a negative test: denied access, misconfiguration, or failed dependency—capture evidence.

Extended practice scenario 6 — 62 kubernetes operations and gitops on eks

Estimate cost dimensions for this lesson's activities. Label figures illustrative; verify pricing pages with dates.

Extended practice scenario 7 — 62 kubernetes operations and gitops on eks

Produce a cleanup checklist and execute destroy for EKS/node groups/LBs created for lesson 62—or document explicit design-only waiver with budget rationale.

Extended practice scenario 8 — 62 kubernetes operations and gitops on eks

Cross-link prior course lessons that this design depends on; list two failure modes inherited from those layers.

Extended practice scenario 9 — 62 kubernetes operations and gitops on eks

Security review: encryption at rest/in transit, public exposure checks, CloudTrail visibility for management events.

Extended practice scenario 10 — 62 kubernetes operations and gitops on eks

Reliability review: failure isolation, rollback, how detection and recovery work under Monday traffic.

Extended practice scenario 11 — 62 kubernetes operations and gitops on eks

Performance review: bottlenecks, scaling limits, or saturation signals relevant to this lesson's topic.

Extended practice scenario 12 — 62 kubernetes operations and gitops on eks

Write a runbook section for on-call: symptoms, first three diagnostic steps, escalation, rollback.

Extended practice scenario 13 — 62 kubernetes operations and gitops on eks

Config/compliance angle: which controls must be true before production promotion.

Extended practice scenario 14 — 62 kubernetes operations and gitops on eks

Multi-account consideration: how this control behaves across sandbox, staging, and production accounts.

Extended practice scenario 15 — 62 kubernetes operations and gitops on eks

Interview drill: explain the lesson topic to a hiring manager in three minutes without inventing APIs.

Extended practice scenario 16 — 62 kubernetes operations and gitops on eks

Pair review: partner stops any invented CLI flags, hardcoded secrets, or fake current prices.

Extended practice scenario 17 — 62 kubernetes operations and gitops on eks

FinOps continuity from lesson 47: idle resources, data transfer, and when to decommission lab stacks.

Extended practice scenario 18 — 62 kubernetes operations and gitops on eks

DR continuity from lesson 45: ensure this design does not destroy recoverability during change.

Extended practice scenario 19 — 62 kubernetes operations and gitops on eks

Produce evidence pack: diagrams, configs, test results, cleanup proof.

Extended practice scenario 20 — 62 kubernetes operations and gitops on eks

Challenge constraint: tighter monthly budget—what do you sequence differently?

Extended practice scenario 21 — 62 kubernetes operations and gitops on eks

Challenge constraint: stricter compliance—what evidence gates release?

Extended practice scenario 22 — 62 kubernetes operations and gitops on eks

Challenge constraint: shorter maintenance window—what pre-staging changes?

Extended practice scenario 23 — 62 kubernetes operations and gitops on eks

Teach-back: recite the Lesson Overview paragraph, then check against the text; flashcard gaps.

Extended practice scenario 24 — 62 kubernetes operations and gitops on eks

Dress rehearsal: tabletop under a timer with observer notes.

Extended practice scenario 25 — 62 kubernetes operations and gitops on eks

Final synthesis note: one paragraph connecting this lesson's topic to TechLearningPro production ownership.

Compliance evidence pack for auditors / platform leads

Budget configs, tagging proof (Project=tlp-lesson62), architecture diagram, ADR, negative-test evidence, encryption settings notes, CloudTrail/Config/SSM references where applicable, pricing-page retrieval dates for any dollar claims, cleanup confirmation.

Glossary drill

Define readiness probe, PDB, GitOps, managed node group, digest pin, IRSA from memory.

Budget and cleanup sworn statement

I verified budgets remain enabled with alert subscribers. I deleted or scheduled deletion for orphan lab resources from lesson 62 named ____. I labeled every savings or cost figure with pricing-page retrieval dates or marked it illustrative. Signed ________ date ________.

Interview story to memorize

Craft a 60-second story: you moved TechLearningPro to GitOps on EKS, pinned digests, added PDBs, rolled back via git revert, and deleted the lab cluster the same day.

Self-assessment rubric

Area	Weight
Conceptual accuracy and service selection	20%
Working design/lab and reproducibility	25%
Security and least-privilege	20%
Reliability, observability, troubleshooting	20%
Documentation, ADR, cleanup	15%

Final synthesis before summary

EKS operations excellence is probes, PDBs, upgrades, GitOps drift control, and ruthless cost cleanup—not a green cluster console tile.

Extended facilitation script (print for guild leads)

Open with cost of an idle EKS lab and an upgrade outage without PDBs. Ask where desired state lives. Assign delete owners for Project=tlp-lesson62 clusters before the meeting ends.

Lesson Summary

Kubernetes operations on EKS for TechLearningPro means probes, resources, PDBs, node/control-plane upgrade discipline, Helm packaging awareness, and GitOps reconciliation with digest pins—plus budgets that force cluster deletion. Deepen API theory in /learn/kubernetes. Next: Production Readiness Reviews and Runbooks.

Kubernetes Operations and GitOps on EKS

Kubernetes Operations and GitOps on EKS

Lesson Overview

Learning Objectives

Prerequisites

Detailed Core Concepts

Operations versus cluster creation

Probes

Requests, limits, and scheduling

PodDisruptionBudgets (PDBs)

Node groups and upgrades

Autoscaling concepts

Helm

GitOps (Argo CD / Flux concepts)

Drift

Architecture and Design Diagrams

Technical Explanation

Rollback story

Upgrade story

IRSA reminder

Cost

Cross-links

Extended depth: GitOps operating model for TechLearningPro

Extended depth: upgrade rehearsal checklist

Extended depth: cost kill-switch

Extended depth: cross-link without cloning

Practical Implementation or Design Walkthrough

Problem

Steps

Enterprise Real-World Scenarios

kubectl snowflakes

No PDBs

latest tag

Cluster left on

Control plane only upgrade

Best Practices

Common Mistakes and Anti-Patterns

Troubleshooting and Design Review

Interview Questions and Answers

Hands-On Exercises

Exercise: GitOps deploy with rollback and drift correction

Challenge ADR

Lab evidence checklist

Narrative: PDB saves the upgrade

Continuity

Extended practice scenario 1 — 62 kubernetes operations and gitops on eks

Extended practice scenario 2 — 62 kubernetes operations and gitops on eks

Extended practice scenario 3 — 62 kubernetes operations and gitops on eks

Extended practice scenario 4 — 62 kubernetes operations and gitops on eks

Extended practice scenario 5 — 62 kubernetes operations and gitops on eks

Extended practice scenario 6 — 62 kubernetes operations and gitops on eks

Extended practice scenario 7 — 62 kubernetes operations and gitops on eks

Extended practice scenario 8 — 62 kubernetes operations and gitops on eks

Extended practice scenario 9 — 62 kubernetes operations and gitops on eks

Extended practice scenario 10 — 62 kubernetes operations and gitops on eks

Extended practice scenario 11 — 62 kubernetes operations and gitops on eks

Extended practice scenario 12 — 62 kubernetes operations and gitops on eks

Extended practice scenario 13 — 62 kubernetes operations and gitops on eks

Extended practice scenario 14 — 62 kubernetes operations and gitops on eks

Extended practice scenario 15 — 62 kubernetes operations and gitops on eks

Extended practice scenario 16 — 62 kubernetes operations and gitops on eks

Extended practice scenario 17 — 62 kubernetes operations and gitops on eks

Extended practice scenario 18 — 62 kubernetes operations and gitops on eks

Extended practice scenario 19 — 62 kubernetes operations and gitops on eks

Extended practice scenario 20 — 62 kubernetes operations and gitops on eks

Extended practice scenario 21 — 62 kubernetes operations and gitops on eks

Extended practice scenario 22 — 62 kubernetes operations and gitops on eks

Extended practice scenario 23 — 62 kubernetes operations and gitops on eks

Extended practice scenario 24 — 62 kubernetes operations and gitops on eks

Extended practice scenario 25 — 62 kubernetes operations and gitops on eks

Compliance evidence pack for auditors / platform leads

Glossary drill

Budget and cleanup sworn statement

Interview story to memorize

Self-assessment rubric

Final synthesis before summary

Extended facilitation script (print for guild leads)

Lesson Summary

Further Reading