S3

Object storage for document binaries, reports, and static assets.

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Sentinel API   │     │  Report Gen     │     │  Terraform CI   │
│   (boto3)       │     │   (boto3)       │     │   (awscli)      │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         │ presigned URL         │ multipart upload      │ state lock
         │ multipart upload      │                       │
         ▼                       ▼                       ▼
┌─────────────────────────────────────────────────────────────────────┐
│                              S3 Buckets                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌─────────┐  │
│  │ cwt-*-uploads│  │ cwt-*-reports│  │ cwt-*-assets │  │tf-state │  │
│  │  (7yr retention)  │ (3yr retention) │  (Indefinite) │ (Versioned)│
│  └──────────────┘  └──────────────┘  └──────────────┘  └─────────┘  │
└─────────────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────┐
│   CloudFront    │
│  (OAC / OAI)    │
└─────────────────┘

Buckets

Bucket	Purpose	Lifecycle
`cwt-{env}-uploads`	Document binaries	7-year retention
`cwt-{env}-reports`	Generated PDF/PPTX/Excel	3-year retention
`cwt-{env}-assets`	Static assets	Indefinite
`terraform-iac-data`	Terraform state	Versioned

Connection Patterns & Client Libraries

Primary SDK: boto3 (Python) — version >=1.34.0

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    config=Config(
        max_pool_connections=50,
        retries={"max_attempts": 3, "mode": "adaptive"}
    )
)

Presigned URLs (preferred for client-side uploads/downloads):

s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": bucket, "Key": key},
    ExpiresIn=300
)

Multipart Upload (required for objects > 100 MB):

Part size: 100 MB
Max parts: 10,000
Use UploadPart + CompleteMultipartUpload for resumable uploads.

Static Asset Delivery:

CloudFront Origin Access Control (OAC) enforces bucket-level authentication.
Assets are referenced via CloudFront domain with signed cookies for private content.

Security

Public access: Blocked on all buckets
Encryption: AES-256 (SSE-S3)
Versioning: Enabled on uploads and reports
CloudFront OAC: For static asset delivery
Bucket policies: Restrict to VPC endpoints and CloudFront OAI/OAC only
Access logging: Enabled on cwt-{env}-uploads to cwt-{env}-access-logs prefix

Backup & Disaster Recovery

Strategy	Scope	RPO	RTO
Cross-Region Replication (CRR)	`uploads`, `reports`	Near-real-time	< 1 hour
S3 Versioning	All buckets	Zero	Minutes
S3 Batch Operations	Bulk restore from Glacier	N/A	Hours
Terraform State	`terraform-iac-data`	Zero	Minutes (DynamoDB lock recovery)

DR Runbook:

Verify replication status via ReplicationStatus head-object check.
Failover: Update DNS / CloudFront origin to DR region bucket.
For Glacier restores, initiate RestoreObject with Tier=Standard (3–5 hours) or Tier=Expedited (1–5 minutes, provisioned capacity required).

Performance Tuning

Technique	Recommendation
Transfer Acceleration	Enabled for `uploads` bucket (global advisor access)
Multipart Threshold	Use multipart for all objects > 100 MB
CloudFront Caching	Cache static assets with `max-age=86400`
S3 Select	Use for CSV/JSON filtering to reduce egress
Request Rate	Prefix keys with UUID or timestamp to avoid hot partitions

Monitoring & Alerting

CloudWatch Metrics:

Metric	Threshold	Severity
`4xxErrors`	> 1% of total requests	Warning
`5xxErrors`	> 0.1% of total requests	Critical
`TotalRequestLatency`	p99 > 500 ms	Warning
`BucketSizeBytes`	> 80% of projected capacity	Warning
`NumberOfObjects`	Growth > 20% week-over-week	Info

Log Analytics:

S3 Server Access Logs → Athena table for ad-hoc querying.
CloudFront access logs → QuickSight dashboard for geographic latency.

S3

S3