Skip to content

Amazon S3

S3 is object storage: buckets contain objects, each identified by a key, at virtually unlimited scale and very high durability. It is not a filesystem — there are no real directories. A key like uploads/2026/report.pdf is a single flat string; the / characters just let consoles and tools simulate a folder hierarchy by grouping on key prefixes.

Storage Classes and Lifecycle Rules

Objects can live in different storage classes trading cost against retrieval speed/frequency — Standard (frequent access), Infrequent Access, Glacier (archival, slow/cheap retrieval), and more. Lifecycle rules automate moving objects between classes, or expiring them entirely, based on age — e.g. "move to Infrequent Access after 30 days, delete after a year" — without any application code needing to manage that transition.

Access Control

Three overlapping mechanisms exist; in practice, bucket policies and IAM policies are what modern setups should use, with ACLs largely legacy:

  • IAM policies — attached to a user/role, granting that principal access to specific S3 actions/resources — the same mechanism used everywhere else in AWS (e.g. a Lambda's execution role).
  • Bucket policies — attached directly to the bucket, controlling who (which principals, possibly public) can access it.
  • Block Public Access — an account/bucket-level setting that overrides any policy attempting to grant public access, a safety net against accidental public exposure — leave it enabled unless a bucket specifically needs to serve public content.

Presigned URLs

A presigned URL grants temporary, time-limited access to a specific private object, generated server-side using AWS credentials, without exposing those credentials to whoever receives the URL:

import boto3

s3 = boto3.client("s3")
url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": "uploads", "Key": "user-123/photo.jpg"},
    ExpiresIn=300,  # seconds
)

This is the standard pattern for letting a client upload/download directly to/from S3 — the backend issues a short-lived URL instead of proxying the file through itself, and the URL stops working once it expires.

Event Notifications

S3 can publish an event when an object is created, deleted, or restored, delivered to Lambda, SQS, or SNS — the standard way to trigger processing the moment a file lands (e.g. resize an uploaded image, parse an uploaded document) instead of polling the bucket for new objects.

Versioning

Enabling versioning on a bucket keeps every previous version of an object instead of overwriting it in place — protection against accidental deletes or overwrites, at the cost of storing every version until explicitly cleaned up (often paired with a lifecycle rule to expire old versions after some time).

Consistency

S3 provides strong read-after-write consistency for all operations (as of a 2020 change) — a GET immediately after a successful PUT is guaranteed to return the new data. Older material describing S3 as "eventually consistent" predates this and no longer applies.

Summary

  • Keys are flat strings; the folder-like hierarchy in consoles/tools is a prefix convention, not a real filesystem.
  • Lifecycle rules automate cost-driven storage-class transitions and expirations without application code.
  • Use IAM/bucket policies for access control; keep Block Public Access on unless a bucket genuinely needs to serve public content.
  • Presigned URLs let clients upload/download directly to/from S3 without a backend proxying the bytes or exposing credentials.
  • Event notifications trigger downstream processing (commonly Lambda or SQS) the moment an object changes.
  • S3 is strongly consistent for all operations today.
  • AWS Lambda — a common consumer of S3 event notifications.
  • AWS Cognito — Identity Pool credentials are frequently scoped to allow direct client uploads to a specific S3 prefix.