AWS Cloud & DevOps Tutorial 0/70 lessons ~6 min read Lesson 9

    Storage Services: S3, EBS and EFS

    When an Auto Scaling group replaces a progress API instance, learners must not lose videos, captions, certificates, or uploaded assignments. Those bytes never belonged on the insta

    Course progress0%
    Focus
    15 guided sections
    Practice signal
    Examples included
    Career prep
    Interview Q&A included

    Storage Services: S3, EBS and EFS

    AWS Cloud & DevOps Engineering · Lesson 9 · AWS Cloud Foundations

    Lesson Overview

    When an Auto Scaling group replaces a progress API instance, learners must not lose videos, captions, certificates, or uploaded assignments. Those bytes never belonged on the instance’s personality. They belong on storage services chosen for access pattern, durability model, and failure domain. Pick wrong, and Monday’s board is slow, expensive, or brittle. Pick EFS because “shared files feel familiar,” and you may pay for a filesystem you did not need. Pick a single EBS volume as the only copy of learner progress, and a single AZ event becomes a data incident.

    Compute Services: EC2, AMIs and Auto Scaling made hosts replaceable. This lesson places durable and semi-durable storage underneath and beside those hosts: Amazon S3 for object storage, Amazon EBS for block volumes attached to instances, and Amazon EFS for shared POSIX file storage when that model is truly required. You will learn storage classes as concepts without inventing unit prices, encryption defaults you should not turn off, lifecycle thinking, and the important sentence when not to use EFS.

    TechLearningPro’s teaching volumes: course assets and progress exports land in S3; database disks are EBS behind managed databases next lesson; a shared scratch filesystem is rare and must justify itself.

    Learning Objectives

    By the end of this lesson you will be able to:

    1. Contrast object, block, and file storage in AWS terms and access patterns.
    2. Choose S3 for durable shared objects addressed by key, not as a boot disk.
    3. Explain EBS attachment, snapshot, and AZ coupling at foundations level.
    4. Explain EFS as NFS-style shared storage and list cases where it is the wrong tool.
    5. Describe S3 storage class intent (hot, infrequent, archive) without quoting fake prices.
    6. Apply encryption at rest and in transit patterns for each service family.
    7. Design lifecycle rules for progress exports and logs that match retention policy.
    8. Avoid anti-patterns: EBS as the only backup, world-open S3, EFS for static web assets.
    9. Budget and clean up lab buckets, volumes, snapshots, and file systems.
    10. Hand lesson 10 a clear split between files/objects and databases.

    Prerequisites

    • Lesson 8 for instance replaceability and why instance store is not a backup.
    • Lesson 7 for VPC endpoints (S3 gateway) and security groups for EFS mounts.
    • IAM fundamentals for bucket policies versus identity policies.
    • A sandbox budget alert before creating multi-GB test objects or provisioned throughput experiments.

    Detailed Core Concepts

    Three storage models

    Object storage (S3) stores objects in buckets, addressed by keys. You read and write over HTTPS APIs (and SDKs). There is no POSIX filesystem rename semantics across arbitrary prefixes the way a local disk works—though many apps still use S3 successfully for assets, data lakes, backups, and static sites. Durability and availability are designed at regional scale for the service; you still configure versioning, replication, and access carefully.

    Block storage (EBS) provides volumes that attach to EC2 instances as block devices. File systems and databases sit on top. A volume lives in a single Availability Zone. Snapshots go to S3-backed snapshot storage for backup and AMI creation. Performance depends on volume type and size/IOPS configuration—verify current volume types (such as gp3) in documentation.

    File storage (EFS) provides a managed NFS filesystem you mount on many instances across AZs in a Region. It fits shared POSIX workloads: lift-and-shift apps that need a shared /mnt/content, home directories, or certain CMS patterns. It is not automatically the right place for cloud-native TechLearningPro assets that can be objects in S3.

    S3 for TechLearningPro

    Use S3 for course videos metadata sidecars, captions, certificate PDFs, static lesson assets, progress export dumps, ALB logs, and AMI-adjacent artifacts like build tarballs. The progress API should fetch or stream via application logic and CloudFront later—not by mounting S3 as a block device (it is not one).

    Bucket design: private by default, Block Public Access on unless you have a deliberate static website or public asset exception with account controls. Separate buckets or prefixes for raw uploads, published assets, and logs. Use IAM roles on EC2/ECS/Lambda to access specific prefixes. Bucket policies add resource-based constraints.

    Versioning protects against delete and overwrite mistakes. It increases storage; lifecycle then expires noncurrent versions. Object Lock appears in compliance stories—mention, do not fake legal hold requirements.

    Storage classes (concepts only): Standard for frequent access; infrequent access classes for objects touched rarely but needed quickly; archive classes for long retention with restore times. Intelligent-Tiering moves objects based on access patterns with monitoring considerations. Exact retrieval times and fees change—read the current storage class documentation and price card. This course will not invent dollar figures. Decision rule: classify by access frequency and restore tolerance, then look up cost.

    Lifecycle rules transition or expire objects. Example intent: progress debug dumps expire after 30 days; noncurrent versions expire after 90; move cold completed-course archives to an infrequent or archive class after 180 days if product agrees. Lifecycle is how S3 bills stay honest.

    Encryption: SSE-S3 or SSE-KMS for at rest; TLS in transit. Bucket keys and KMS design deepen in later security lessons. Default encryption should be on. Deny unencrypted puts with policies when your organization requires it.

    S3 Gateway VPC endpoint keeps private subnet traffic to S3 on the AWS network without NAT for that path. Pair with bucket policies that restrict access to the VPC endpoint where appropriate.

    EBS for disks that boot and data that sits beside an instance

    Root volumes boot the OS. Additional volumes hold application data for patterns that need block storage. Snapshots are point-in-time backups that are incremental behind the scenes. Restore creates a new volume in an AZ you choose. Encryption should be enabled; copying snapshots across Regions is a DR tool with transfer and storage cost.

    AZ coupling: an EBS volume in AZ-a cannot attach to an instance in AZ-b. Multi-AZ high availability for a self-managed database means replication at the database layer or using managed RDS (next lesson)—not magical multi-AZ EBS attach for the same volume.

    Volume types: general purpose SSD for most foundations work; io-family for latency-sensitive databases when metrics demand; throughput-optimized HDD for large sequential throughput sequential workloads—verify names and guidance in current docs. Do not over-provision IOPS in a lab “to be safe.”

    Instance store is temporary local disk on some instance types. Fast and ephemeral. Not for learner progress. If the instance stops or fails, treat data as gone.

    EFS: powerful, easy to misuse

    EFS mounts via NFS. Multiple progress API instances can see the same files. That solves lift-and-shift shared content directories. It also introduces lock semantics, mount target security groups, throughput modes, and a cost model driven by storage and sometimes provisioned throughput or requests depending on configuration—again, check current pricing dimensions, do not invent them.

    When not to use EFS:

    • Static assets that S3 + CDN serve better.
    • Database storage (use EBS/RDS/Aurora designs).
    • High-churn temporary scratch that instance store or memory can hold.
    • “We need a shared disk” without a POSIX requirement—often S3 is enough.
    • Cross-Region active-active without understanding that EFS is regional; replication features exist and must be designed, not assumed.

    Prefer S3 for TechLearningPro published course objects unless an application truly needs POSIX sharing.

    Encryption, identity, and network together

    Storage security is three doors: who (IAM/bucket policy), network path (VPC endpoint, mount target SG, TLS), and encryption (at rest keys, in transit). Opening a bucket publicly “for the video player” is rarely needed with CloudFront origin access patterns (later). Mounting EFS with a security group that allows NFS from the entire VPC CIDR repeats the database blast-radius mistake from lesson 7.

    Architecture and Design Diagrams

    mermaid
    flowchart TB
    API["Progress API ASG"]
    S3["S3 course assets + exports"]
    EBS["EBS root on each instance"]
    EFS["EFS only if POSIX shared required"]
    DB["Managed DB on EBS - lesson 10"]
    API -->|"PutObject GetObject"| S3
    API --- EBS
    API -.->|"avoid unless needed"| EFS
    DB --- EBS

    Decision sketch:

    text
    Need POSIX multi-attach across instances? → consider EFS
    Need durable objects via API, lake, backup, static? → S3
    Need disk for OS or single-instance DB engine? → EBS
    Need temporary scratch? → instance store or memory, not “prod EFS”

    Lifecycle flow:

    mermaid
    flowchart LR
    Put["Object written Standard"] --> Hot["Frequent reads"]
    Put --> Life["Lifecycle rule"]
    Life --> IA["Infrequent access class"]
    Life --> Arch["Archive class"]
    Life --> Exp["Expire"]

    Technical Explanation

    Consistency and application behavior

    S3 provides strong read-after-write consistency for new objects and overwrites as documented in current S3 consistency model materials. Applications should still use safe key design and avoid assuming directory listing is a transaction log. List operations and pagination matter for large prefixes. The progress API should store object keys in the database rather than inventing recursive list-as-database anti-patterns.

    Multipart upload and large assets

    Large videos use multipart upload. Incomplete multiparts consume storage until aborted or lifecycle-aborted. Labs that upload big files must abort incomplete uploads and delete objects in cleanup.

    EBS snapshots and AMIs revisited

    Creating an AMI snapshots the root volume. Deregistering an AMI without deleting snapshots leaves cost. Snapshot frequency for self-managed data volumes must match RPO; managed databases automate much of this next lesson. Crash consistency versus application consistency matters for databases—flush or use engine-aware backup.

    EFS mount targets and throughput

    Mount targets appear in subnets; security groups must allow NFS from clients. Throughput modes and performance modes affect scaling behavior—read current EFS guides before setting production. Bursting credits can surprise teams who treat EFS like an infinite local SSD. Measure, then choose modes deliberately.

    Data transfer mental model

    Egress to the internet costs; same-Region S3 access patterns via gateway endpoints reduce NAT dependency. Cross-Region replication and cross-AZ traffic have costs and latency. Lesson 11 frames cost allocation; here, avoid copying terabytes “for safety” without a restore test.

    Practical Implementation or Design Walkthrough

    Problem

    Design storage for TechLearningPro lab: (1) private S3 bucket for assets and a lifecycle rule, (2) confirm EC2 root EBS encryption, (3) optionally create a small EFS and mount from one instance to feel NFS, then decide with a written ADR whether production progress assets would use EFS—expected answer usually no. Budget first; cleanup mandatory.

    Steps

    1. Budget alert + tags Project=tlp-lesson09.
    2. Create a private bucket; enable Block Public Access; enable default encryption; enable versioning for the exercise.
    3. Add a lifecycle rule that expires a tmp/ prefix quickly and aborts incomplete multipart uploads after a short number of days (verify current configuration fields).
    4. From an instance role with least privilege, put and get a small object; prove a deny by removing permission.
    5. Describe the instance volume for encrypted=true.
    6. Optional: create EFS, mount targets in private subnets, SG allowing NFS from sg-app only, mount, write a file, unmount.
    7. Write the ADR: S3 for assets; EBS for roots; EFS not for static course objects.
    8. Cleanup: delete objects and bucket (versioned buckets need version deletes), delete EFS, delete unused volumes/snapshots from the lab, terminate instances.

    Evidence

    CLI or console proof of Block Public Access, encryption, lifecycle JSON/document, IAM deny test, cleanup confirmation.

    Enterprise Real-World Scenarios

    TechLearningPro: EFS for everything

    A team mounts EFS for videos, captions, and user uploads because the legacy LMS used NFS. Latency and cost climb; scaling the API multiplies client connections. Migration moves published assets to S3, keeps a temporary EFS only for a legacy report job that needs POSIX, and deletes the catch-all filesystem after the job is rewritten.

    Public bucket incident

    An uploads bucket was public for “easier QA.” Search engines indexed learner submissions. Fix: Block Public Access at account and bucket; CloudFront or signed URLs for legitimate delivery; access logging and a bucket inventory; incident review that treats public education data as a severity-1 class event.

    Snapshot sprawl

    Daily AMIs from lesson 8 labs left dozens of snapshots. Finance asks why storage grew. Fix: deregister lab AMIs, delete snapshots, lifecycle on noncurrent S3 versions, and a weekly orphan snapshot report tagged by Project.

    Best Practices

    • Private buckets by default; Block Public Access on.
    • Encrypt everything at rest; TLS in transit.
    • Separate prefixes/buckets for logs, publishes, and uploads.
    • Lifecycle for expire and abort incomplete multipart.
    • Prefer S3 for shared durable assets; EFS only with POSIX need.
    • Encrypt EBS; snapshot with a tested restore.
    • Do not treat instance store as durable.
    • Least privilege prefix-level IAM for instance roles.
    • Use S3 gateway endpoints from private subnets when suitable.
    • Tag buckets, volumes, EFS, and snapshots for cost.
    • Version important buckets; expire noncurrent versions intentionally.
    • Document RPO for snapshot schedules on self-managed disks.
    • Clean versioned buckets carefully during lab teardown.
    • Prefer CloudFront + S3 for public learning assets later rather than public buckets.
    • Review EFS security groups as carefully as database groups.

    Common Mistakes and Anti-Patterns

    Public write or list on educational content buckets. Data leak. Block public access.

    EFS for static video. Wrong tool. Use S3.

    Single EBS volume as only backup of critical data. AZ failure risk. Snapshot + multi-AZ data services.

    Forgetting incomplete multipart cleanup. Quiet storage cost.

    World-open NFS on EFS. Blast radius. SG from app tier only.

    Unencrypted volumes “temporary.” Temporary becomes permanent.

    Using list-of-bucket as a database. Slow and racy. Store keys in a real database.

    Cross-Region copy without restore test. Hope is not DR.

    Leaving lab EFS and buckets after class. Budget pain.

    Assuming EBS multi-attach solves multi-AZ HA for arbitrary filesystems. Understand limits; prefer designed data services.

    Troubleshooting and Design Review

    AccessDenied on GetObject. IAM identity policy, bucket policy, VPC endpoint policy, KMS key policy, object ownership/ACLs legacy issues.

    Slow S3 downloads in app subnet. Missing gateway endpoint forcing NAT; or wrong Region; or tiny instance network.

    Cannot attach volume. Wrong AZ; volume in-use; encryption permission.

    EFS mount hang. SG, mount target subnet route, NFS client helpers, network ACLs.

    Bucket delete fails. Versioning; delete markers; object lock.

    Design review: Is every durable learner object in S3 or a database? Any EFS justification written? Encryption on? Lifecycle matches retention? Lab cleaned?

    Interview Questions and Answers

    Q1. S3 vs EBS vs EFS in one minute?

    S3 is object API storage for durable shared objects. EBS is AZ-scoped block disks for instances. EFS is managed NFS shared across instances in a Region. Choose by access pattern.

    Q2. When is EFS the wrong choice?

    When objects can be stored in S3, when you need a database, or when you only wanted “shared storage” without POSIX semantics.

    Q3. Why can’t an EBS volume move freely across AZs while attached?

    Volumes are AZ resources. Cross-AZ use means snapshot/restore or design replication at another layer.

    Q4. What does S3 versioning buy you?

    Recovery from overwrite and delete mistakes, at the cost of storing noncurrent versions until lifecycle expires them.

    Q5. Name storage class decision drivers without prices.

    Access frequency, retrieval time tolerance, durability needs, and operational complexity of restores.

    Q6. How do you stop accidental public buckets?

    Account and bucket Block Public Access, IAM Guardrails/SCPs in organizations, private ACLs disabled patterns, continuous scanning.

    Q7. What is a gateway endpoint for S3?

    A route-table entry that keeps S3 traffic on the AWS network from your VPC without requiring a NAT Gateway for that traffic path.

    Q8. How do snapshots relate to AMIs?

    AMIs reference snapshots of root (and optionally other) volumes. Deleting carefully requires understanding both objects.

    Q9. Incomplete multipart uploads matter because?

    They consume storage until aborted; lifecycle rules should abort them.

    Q10. How should the progress API authorize S3 access?

    Instance/task role with least privilege to specific bucket prefixes; no embedded keys; optional bucket policy conditions.

    Q11. EFS security group baseline?

    Allow NFS from application security groups only, not from 0.0.0.0/0 or entire VPC unless justified.

    Q12. Instance store versus EBS?

    Instance store is ephemeral local disk; EBS persists independently of instance lifetime (until deleted) and supports snapshots.

    Hands-On Exercises

    Exercise: Assets on S3, disks understood, EFS optional

    Problem statement. Build the private asset bucket with encryption, versioning, lifecycle, and IAM allow/deny tests. Document EBS encryption on a lab instance. Optional EFS mount. ADR on when not to use EFS. Budget + cleanup including versioned deletes.

    Rubric.

    Assessment areaWeight
    S3 security baseline25%
    Lifecycle and multipart abort15%
    EBS encryption awareness10%
    EFS decision quality20%
    Deny-path IAM test15%
    Budget and cleanup15%

    Challenge

    Design storage for application assets, database disks, and a legacy shared report directory. Minimize EFS. State encryption and lifecycle for each store.

    Deep dive: object key design for progress and courses

    TechLearningPro keys should encode a stable hierarchy without pretending S3 is a relational database. A workable pattern is org/{orgId}/course/{courseId}/lesson/{lessonId}/asset/{assetId}.{ext} for published assets and org/{orgId}/uploads/{yyyy}/{mm}/{uuid} for raw uploads that a processor promotes. The database stores the key and the content hash. Listing a prefix is for operations and inventory, not for every learner page view. Hot path reads use GetObject with the key from the database or a signed URL. Hot path lists invite throttling and slow Monday boards.

    Avoid unbounded unique prefixes that defeat S3 partition scaling myths of old, and also avoid single hot keys that every instance stampedes. Current S3 scales strongly, but application design still matters: cache metadata, do not list millions of keys on request. For exports, write exports/{date}/progress-{shard}.json.gz and record completion in the database so the board never guesses from partial lists.

    Rename semantics differ from filesystems. Overwrite is a new version if versioning is on. “Move” is copy plus delete. Multi-step moves need application-level checks. Teach engineers not to build transactional workflows that assume POSIX rename atomicity across prefixes unless they have designed for it.

    Deep dive: encryption choices without theater

    SSE-S3 encrypts with keys managed by S3 and is often enough for many course assets. SSE-KMS adds an audit trail in CloudTrail for key use and centralized key policy control, at the cost of KMS request patterns and key administration. SSE-C exists for customer-provided keys and is rarely what a foundations team should choose for TechLearningPro without a hard requirement. Dual-layer encryption options appear in documentation for specialized needs—verify current options rather than memorizing marketing names.

    In transit, require HTTPS for all S3 access from applications. For EFS, use TLS mount options as documented. For EBS, encryption at rest protects volumes and snapshots; ensure the instance role and account defaults enforce encrypted volume creates so a tired operator cannot launch cleartext disks. KMS key policies can deny deletes or restrict admins; Module 4’s KMS lesson expands this. Foundations rule: encryption on, public access off, policies least privilege.

    Deep dive: lifecycle as product policy

    Lifecycle is not a finance trick first; it is product retention expressed as automation. If learner upload drafts are worthless after thirty days, expire them. If certificate PDFs must remain for seven years, do not expire them in a lab-like rule copied from a blog. If debug progress dumps are large, put them under a prefix with a short expiry. Transition to infrequent or archive classes only when restore time is acceptable for the use case. Archive restore for a Monday incident is a bad surprise if nobody tested restore.

    Versioned buckets need noncurrent version expiration. Delete markers can accumulate. Operations should understand how to permanently delete when the policy allows. Object Lock in compliance mode prevents deletion until retain-until-date—powerful and dangerous for labs. Do not enable Object Lock on a sandbox bucket unless you accept the retention consequences.

    Incomplete multipart upload abort rules are mandatory hygiene for video classrooms. A failed 2 GB upload left incomplete ten thousand times is a storage incident with an educational costume.

    Deep dive: EBS performance literacy

    General purpose SSD volumes suit roots and many application disks. Size and IOPS/throughput configuration on modern types like gp3 let you tune without oversizing capacity solely for performance—confirm current mechanics in the EBS user guide. Burst behavior on older patterns surprised teams; measure CloudWatch volume metrics before declaring the database “slow” and upgrading the instance type alone.

    Snapshots are incremental, but the first snapshot of a large volume copies a lot; plan time. Restoring a snapshot to a new volume and attaching it is a rehearsal you should do before you need it. Cross-AZ recovery means restoring in the target AZ and pointing the application at it, or better, using multi-AZ managed databases next lesson. Encrypting snapshots and sharing them across accounts for DR pipelines is an enterprise pattern; permissions must be least privilege.

    Delete-on-termination for roots is usually correct for ASG instances. Data volumes with critical self-managed state should be carefully managed so ASG scale-in does not destroy the only copy. Prefer managed data services for learner progress.

    Deep dive: EFS failure modes and cost drivers

    EFS shines when many instances need the same POSIX tree. It hurts when every static asset becomes a small file under heavy metadata operations, or when throughput mode is misunderstood. Watch for security groups that open NFS widely. Watch for a single Region filesystem assumed to be multi-Region. Watch for backups: AWS Backup or EFS backup features should be deliberate. Watch for mounting from public subnets without need.

    Cost drivers typically include storage consumed and, depending on configuration, throughput provisioning or request characteristics—read the current price dimensions. The design move that usually saves TechLearningPro money is moving static and large media to S3 and leaving EFS only for the stubborn POSIX app. “We already know NFS” is not an architecture decision record.

    Scenario: certificate PDF pipeline

    Certificates generate as PDFs when a learner completes a course. A worker writes to s3://…/certs/{learnerId}/{courseId}.pdf with SSE-KMS, records the key in the database, and the learner downloads via a short-lived signed URL. No public bucket. No EFS. No EBS share. If the worker runs on Spot, the PDF still lands in S3 before the job reports success. Monday’s “I lost my certificate” ticket becomes a GetObject permission or key mismatch issue, not an instance replacement tragedy.

    Scenario: lift-and-shift CMS

    An acquired company runs a CMS that stores templates and uploads on NFS. During migration, EFS provides a transitional shared filesystem while the progress API remains on ASG. Parallel work streams new uploads to S3. After feature parity, EFS shrinks to read-only historical files, then a freeze date, then decommission. The ADR states the freeze date and the owner. Without the freeze date, EFS remains forever because nobody wants to be the person who deletes nostalgia.

    Scenario: ransomware-minded protections

    Versioning plus MFA delete or Object Lock (where appropriate), restricted IAM, CloudTrail data events for sensitive buckets, and offline or replicated backups form a layered response. Foundations introduces the idea: a single delete API call should not silently vaporize the only copy of course videos. Exact control combinations belong to security architecture reviews. Labs should still practice versioning and recovery of a deleted object.

    Lab narrative: proving deny and cleanup on versioned buckets

    You enable versioning, put tmp/a.txt, put again, delete, see a delete marker, restore by deleting the delete marker or getting a noncurrent version—follow current console/CLI guidance. Then you attempt cleanup: deleting a versioned bucket requires emptying all versions. Document the commands you used. Teams that skip this learn during automation when aws s3 rb --force expectations meet versioning reality. Include abort of multipart uploads in the same cleanup script.

    Interview expansion: durability versus backup

    Service durability targets are not the same as your backup strategy. S3’s durability design does not replace versioning and cross-account or cross-Region copies for ransomware and account-level disasters. EBS durability in an AZ does not replace snapshots. EFS durability does not replace application-aware recovery tests. Always ask: what is the restore unit, who runs the restore, and when did we last practice?

    Connecting storage to Monday reliability

    The board reads aggregates from databases and caches next lesson, but the media and exports behind lessons hit S3. If the instance role lost s3:GetObject after a template change, pages degrade even when EC2 health checks pass. Include a synthetic check that fetches a canary object. If EFS was wrongly on the path for every asset, a filesystem credit or mount issue can take the whole fleet down together—another reason to prefer S3 for shared assets so failures isolate better through HTTP semantics and retries.

    Hands-on hardening checklist for storage labs

    1. Budget alert exists. 2. Block Public Access on. 3. Default encryption on. 4. Versioning decision recorded. 5. Lifecycle matches retention. 6. IAM least privilege tested with deny. 7. VPC endpoint considered for S3. 8. EFS SG locked if EFS used. 9. EBS encrypted. 10. Cleanup empties versions, aborts multipart, deletes EFS, snapshots, and buckets. 11. ADR written for EFS avoidance. 12. Tags present for cost allocation.

    Technical note: S3 Inventory and Storage Lens awareness

    At enterprise scale, inventory reports and Storage Lens dashboards reveal prefixes that grow without owners. Foundations does not require enabling them in every lab, but interview and design review answers should know they exist for cost and hygiene. A weekly inventory of the uploads prefix often finds abandoned multipart and forgotten QA buckets faster than memory.

    Technical note: mount helpers and fstab discipline

    EFS mounts that rely on a human remounting after reboot will fail ASG replacements. Use AMI baking or user data that mounts idempotently with documented TLS and IAM authorization features where applicable. Prefer applications that stop needing boot-time mounts. If user data mounts EFS before the network is ready, reboot loops occur—retry with backoff.

    Anti-pattern clinic: database files on EFS

    Someone places PostgreSQL data files on EFS for “shared storage HA.” This misunderstands database storage, locking, latency, and failover. Use RDS/Aurora or properly designed disk layouts. EFS is not a shortcut to multi-AZ relational databases. Lesson 10 exists so you do not invent one on a network filesystem.

    Anti-pattern clinic: world-readable educational recordings

    Recorded office hours accidentally land in a public prefix. Fix is technical and process: private buckets, signed delivery, training for instructors, scanning for public objects, and rapid containment. Storage configuration is part of learner trust.

    Cost hygiene without fake prices

    Meters you will see: S3 storage by class, requests, data transfer, retrieval for cold classes, EBS GB-month and IOPS/throughput features, snapshot storage, EFS storage and throughput-related charges, NAT bytes if you skipped endpoints. Exact numbers live on the price card. Design choices that usually dominate for TechLearningPro: storing many copies of video in hot class forever; NAT for S3; forgotten snapshots; EFS for static media. Fix classes, endpoints, lifecycle, and tool choice before negotiating discounts.

    Design review script

    Ask: Where does each durable byte live? What is the restore test date? Is any filesystem share required by POSIX? Are buckets private and encrypted? Do lifecycle rules match legal retention? Do ASG instances still work if EFS is deleted? If yes to the last question for asset delivery, you are closer to cloud-native. If no, you have a hard dependency to monitor like a database.

    Additional operational note 1

    Deep dive: object key design for progress and courses

    TechLearningPro keys should encode a stable hierarchy without pretending S3 is a relational database. A workable pattern is org/{orgId}/course/{courseId}/lesson/{lessonId}/asset/{assetId}.{ext} for published assets and org/{orgId}/uploads/{yyyy}/{mm}/{uuid} for raw uploads that a processor promotes. The database stores the key and the content hash. Listing a prefix is for operations and inventory, not for every learner page view. Hot path reads use GetObject with the key from the database or a signed URL. Hot path lists invite throttling and slow Monday boards.

    Avoid unbounded unique prefixes that defeat S3 partition scaling myths of old, and also avoid single hot keys that every instance stampedes. Current S3 scales strongly, but application design still matters: cache metadata, do not list millions of keys on request. For exports, write exports/{date}/progress-{shard}.json.gz and record completion in the database so the board never guesses from partial lists.

    Rename semantics differ from filesystems. Overwrite is a new version if versioning is on. “Move” is copy plus delete. Multi-step moves need application-level checks. Teach engineers not to build transactional workflows that assume POSIX rename atomicity across prefixes unless they have designed for it.

    Additional operational note 2

    Deep dive: encryption choices without theater

    SSE-S3 encrypts with keys managed by S3 and is often enough for many course assets. SSE-KMS adds an audit trail in CloudTrail for key use and centralized key policy control, at the cost of KMS request patterns and key administration. SSE-C exists for customer-provided keys and is rarely what a foundations team should choose for TechLearningPro without a hard requirement. Dual-layer encryption options appear in documentation for specialized needs—verify current options rather than memorizing marketing names.

    In transit, require HTTPS for all S3 access from applications. For EFS, use TLS mount options as documented. For EBS, encryption at rest protects volumes and snapshots; ensure the instance role and account defaults enforce encrypted volume creates so a tired operator cannot launch cleartext disks. KMS key policies can deny deletes or restrict admins; Module 4’s KMS lesson expands this. Foundations rule: encryption on, public access off, policies least privilege.

    Additional operational note 3

    Deep dive: lifecycle as product policy

    Lifecycle is not a finance trick first; it is product retention expressed as automation. If learner upload drafts are worthless after thirty days, expire them. If certificate PDFs must remain for seven years, do not expire them in a lab-like rule copied from a blog. If debug progress dumps are large, put them under a prefix with a short expiry. Transition to infrequent or archive classes only when restore time is acceptable for the use case. Archive restore for a Monday incident is a bad surprise if nobody tested restore.

    Versioned buckets need noncurrent version expiration. Delete markers can accumulate. Operations should understand how to permanently delete when the policy allows. Object Lock in compliance mode prevents deletion until retain-until-date—powerful and dangerous for labs. Do not enable Object Lock on a sandbox bucket unless you accept the retention consequences.

    Incomplete multipart upload abort rules are mandatory hygiene for video classrooms. A failed 2 GB upload left incomplete ten thousand times is a storage incident with an educational costume.

    Additional operational note 4

    Deep dive: EBS performance literacy

    General purpose SSD volumes suit roots and many application disks. Size and IOPS/throughput configuration on modern types like gp3 let you tune without oversizing capacity solely for performance—confirm current mechanics in the EBS user guide. Burst behavior on older patterns surprised teams; measure CloudWatch volume metrics before declaring the database “slow” and upgrading the instance type alone.

    Snapshots are incremental, but the first snapshot of a large volume copies a lot; plan time. Restoring a snapshot to a new volume and attaching it is a rehearsal you should do before you need it. Cross-AZ recovery means restoring in the target AZ and pointing the application at it, or better, using multi-AZ managed databases next lesson. Encrypting snapshots and sharing them across accounts for DR pipelines is an enterprise pattern; permissions must be least privilege.

    Delete-on-termination for roots is usually correct for ASG instances. Data volumes with critical self-managed state should be carefully managed so ASG scale-in does not destroy the only copy. Prefer managed data services for learner progress.

    Additional operational note 5

    Deep dive: EFS failure modes and cost drivers

    EFS shines when many instances need the same POSIX tree. It hurts when every static asset becomes a small file under heavy metadata operations, or when throughput mode is misunderstood. Watch for security groups that open NFS widely. Watch for a single Region filesystem assumed to be multi-Region. Watch for backups: AWS Backup or EFS backup features should be deliberate. Watch for mounting from public subnets without need.

    Cost drivers typically include storage consumed and, depending on configuration, throughput provisioning or request characteristics—read the current price dimensions. The design move that usually saves TechLearningPro money is moving static and large media to S3 and leaving EFS only for the stubborn POSIX app. “We already know NFS” is not an architecture decision record.

    Additional operational note 6

    Scenario: certificate PDF pipeline

    Certificates generate as PDFs when a learner completes a course. A worker writes to s3://…/certs/{learnerId}/{courseId}.pdf with SSE-KMS, records the key in the database, and the learner downloads via a short-lived signed URL. No public bucket. No EFS. No EBS share. If the worker runs on Spot, the PDF still lands in S3 before the job reports success. Monday’s “I lost my certificate” ticket becomes a GetObject permission or key mismatch issue, not an instance replacement tragedy.

    Additional operational note 7

    Scenario: lift-and-shift CMS

    An acquired company runs a CMS that stores templates and uploads on NFS. During migration, EFS provides a transitional shared filesystem while the progress API remains on ASG. Parallel work streams new uploads to S3. After feature parity, EFS shrinks to read-only historical files, then a freeze date, then decommission. The ADR states the freeze date and the owner. Without the freeze date, EFS remains forever because nobody wants to be the person who deletes nostalgia.

    Additional operational note 8

    Scenario: ransomware-minded protections

    Versioning plus MFA delete or Object Lock (where appropriate), restricted IAM, CloudTrail data events for sensitive buckets, and offline or replicated backups form a layered response. Foundations introduces the idea: a single delete API call should not silently vaporize the only copy of course videos. Exact control combinations belong to security architecture reviews. Labs should still practice versioning and recovery of a deleted object.

    Additional operational note 9

    Lab narrative: proving deny and cleanup on versioned buckets

    You enable versioning, put tmp/a.txt, put again, delete, see a delete marker, restore by deleting the delete marker or getting a noncurrent version—follow current console/CLI guidance. Then you attempt cleanup: deleting a versioned bucket requires emptying all versions. Document the commands you used. Teams that skip this learn during automation when aws s3 rb --force expectations meet versioning reality. Include abort of multipart uploads in the same cleanup script.

    Additional operational note 10

    Interview expansion: durability versus backup

    Service durability targets are not the same as your backup strategy. S3’s durability design does not replace versioning and cross-account or cross-Region copies for ransomware and account-level disasters. EBS durability in an AZ does not replace snapshots. EFS durability does not replace application-aware recovery tests. Always ask: what is the restore unit, who runs the restore, and when did we last practice?

    Additional operational note 11

    Connecting storage to Monday reliability

    The board reads aggregates from databases and caches next lesson, but the media and exports behind lessons hit S3. If the instance role lost s3:GetObject after a template change, pages degrade even when EC2 health checks pass. Include a synthetic check that fetches a canary object. If EFS was wrongly on the path for every asset, a filesystem credit or mount issue can take the whole fleet down together—another reason to prefer S3 for shared assets so failures isolate better through HTTP semantics and retries.

    Additional operational note 12

    Hands-on hardening checklist for storage labs

    1. Budget alert exists. 2. Block Public Access on. 3. Default encryption on. 4. Versioning decision recorded. 5. Lifecycle matches retention. 6. IAM least privilege tested with deny. 7. VPC endpoint considered for S3. 8. EFS SG locked if EFS used. 9. EBS encrypted. 10. Cleanup empties versions, aborts multipart, deletes EFS, snapshots, and buckets. 11. ADR written for EFS avoidance. 12. Tags present for cost allocation.

    Additional operational note 13

    Technical note: S3 Inventory and Storage Lens awareness

    At enterprise scale, inventory reports and Storage Lens dashboards reveal prefixes that grow without owners. Foundations does not require enabling them in every lab, but interview and design review answers should know they exist for cost and hygiene. A weekly inventory of the uploads prefix often finds abandoned multipart and forgotten QA buckets faster than memory.

    Additional operational note 14

    Technical note: mount helpers and fstab discipline

    EFS mounts that rely on a human remounting after reboot will fail ASG replacements. Use AMI baking or user data that mounts idempotently with documented TLS and IAM authorization features where applicable. Prefer applications that stop needing boot-time mounts. If user data mounts EFS before the network is ready, reboot loops occur—retry with backoff.

    Additional operational note 15

    Anti-pattern clinic: database files on EFS

    Someone places PostgreSQL data files on EFS for “shared storage HA.” This misunderstands database storage, locking, latency, and failover. Use RDS/Aurora or properly designed disk layouts. EFS is not a shortcut to multi-AZ relational databases. Lesson 10 exists so you do not invent one on a network filesystem.

    Additional operational note 16

    Anti-pattern clinic: world-readable educational recordings

    Recorded office hours accidentally land in a public prefix. Fix is technical and process: private buckets, signed delivery, training for instructors, scanning for public objects, and rapid containment. Storage configuration is part of learner trust.

    Additional operational note 17

    Cost hygiene without fake prices

    Meters you will see: S3 storage by class, requests, data transfer, retrieval for cold classes, EBS GB-month and IOPS/throughput features, snapshot storage, EFS storage and throughput-related charges, NAT bytes if you skipped endpoints. Exact numbers live on the price card. Design choices that usually dominate for TechLearningPro: storing many copies of video in hot class forever; NAT for S3; forgotten snapshots; EFS for static media. Fix classes, endpoints, lifecycle, and tool choice before negotiating discounts.

    Additional operational note 18

    Design review script

    Ask: Where does each durable byte live? What is the restore test date? Is any filesystem share required by POSIX? Are buckets private and encrypted? Do lifecycle rules match legal retention? Do ASG instances still work if EFS is deleted? If yes to the last question for asset delivery, you are closer to cloud-native. If no, you have a hard dependency to monitor like a database.

    Additional operational note 19

    Deep dive: object key design for progress and courses

    TechLearningPro keys should encode a stable hierarchy without pretending S3 is a relational database. A workable pattern is org/{orgId}/course/{courseId}/lesson/{lessonId}/asset/{assetId}.{ext} for published assets and org/{orgId}/uploads/{yyyy}/{mm}/{uuid} for raw uploads that a processor promotes. The database stores the key and the content hash. Listing a prefix is for operations and inventory, not for every learner page view. Hot path reads use GetObject with the key from the database or a signed URL. Hot path lists invite throttling and slow Monday boards.

    Avoid unbounded unique prefixes that defeat S3 partition scaling myths of old, and also avoid single hot keys that every instance stampedes. Current S3 scales strongly, but application design still matters: cache metadata, do not list millions of keys on request. For exports, write exports/{date}/progress-{shard}.json.gz and record completion in the database so the board never guesses from partial lists.

    Rename semantics differ from filesystems. Overwrite is a new version if versioning is on. “Move” is copy plus delete. Multi-step moves need application-level checks. Teach engineers not to build transactional workflows that assume POSIX rename atomicity across prefixes unless they have designed for it.

    Additional operational note 20

    Deep dive: encryption choices without theater

    SSE-S3 encrypts with keys managed by S3 and is often enough for many course assets. SSE-KMS adds an audit trail in CloudTrail for key use and centralized key policy control, at the cost of KMS request patterns and key administration. SSE-C exists for customer-provided keys and is rarely what a foundations team should choose for TechLearningPro without a hard requirement. Dual-layer encryption options appear in documentation for specialized needs—verify current options rather than memorizing marketing names.

    In transit, require HTTPS for all S3 access from applications. For EFS, use TLS mount options as documented. For EBS, encryption at rest protects volumes and snapshots; ensure the instance role and account defaults enforce encrypted volume creates so a tired operator cannot launch cleartext disks. KMS key policies can deny deletes or restrict admins; Module 4’s KMS lesson expands this. Foundations rule: encryption on, public access off, policies least privilege.

    Lesson Summary

    S3 holds TechLearningPro’s durable objects; EBS boots and disks instances and many database engines; EFS is a deliberate POSIX shared filesystem, not the default shared folder for cloud assets. Encryption, private access, lifecycle, and cleanup are part of storage design. When instances die, storage choices decide whether Monday still has the course.

    Databases and Caching on AWS places structured progress state where files should not pretend to be databases.

    Further Reading

    Ready to mark this lesson complete?Track your journey across the entire course.