Cloud Engineer: S3 Storage Optimization
AnthroposFour S3 buckets and inconsistent conventions make it hard to manage costs and data, while workloads vary between production, archives and logs.
Everything is in S3 Standard, with no lifecycle or versioning, so deletions or overwrites can become incidents and recurring waste.
Your goal is to reconstruct the current state and propose actionable tiering, retention, and versioning, defending the trade-offs with stakeholders.
You are a Junior Cloud Engineer at NovaByte Media, a digital media company on AWS.
NovaByte Media is a fast-growing digital media company serving over 200 clients with media processing pipelines on AWS: EC2 machines for heavy jobs, Lambda that performs resizing/derivatives and writes objects to S3, distribution via CloudFront, and centralized logs in S3. Storage is concentrated in four main buckets: novabyte-production-assets (8 TB of active assets), novabyte-archive (5 TB of projects typically reopened between 90–365 days), novabyte-logs (2 TB with compliance constraints) and novabyte-internal-docs (500 GB). Objects are organized with prefixes like /clients/<client-id>/campaigns/<campaign-id>/(raw|renditions|deliverables) and historical materials under /archives/<year>/; inconsistent conventions over time have made data governance difficult. The Public Access Block is enabled, but currently everything is in S3 Standard with no lifecycle or versioning, so there are no guardrails against deletions or overwrites.
You are a Junior Cloud Engineer and will first conduct a conversational audit with Francesca Rinaldi to reconstruct buckets, volumes, prefixes, access patterns and operational practices that allowed the accidental deletion; then you will discuss an operational proposal with Carlos Rivera to translate the audit into a practical plan for tiering, retention and versioning. In one-on-one chats you must propose, bucket by bucket (or by prefix where relevant), target storage classes or transition sequences, describable lifecycle rules (when to move, when to delete or enforce retention), and an S3 Versioning strategy prioritizing novabyte-production-assets explaining versions and delete markers clearly. Also include rules to manage non-current versions to contain costs and justify trade-offs on latency, retrieval costs and minimum durations of the coldest classes; defend your choices under Carlos's practical questions while keeping the plan operable for daily operations.