Secure Document Storage

S3-compatible encrypted document storage with classification levels and audit logging.

Overview

The secure document storage system provides end-to-end encryption for sensitive files using any S3-compatible object storage (Storj, AWS S3, MinIO) combined with AES-256-GCM encryption.

Key Features: - AES-256-GCM encryption for all non-public documents - Document classification (public, internal, confidential, restricted) - Automatic data retention based on classification level - Complete audit logging with IP tracking - User-specific key derivation - unique keys per user

Architecture

┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐
│   Upload        │────▶│  AES-256-GCM     │────▶│  S3 Storage    │
│   Request       │     │  Encryption      │     │  (Encrypted)   │
└─────────────────┘     └──────────────────┘     └────────────────┘
         │                       │                        │
         ▼                       ▼                        ▼
┌─────────────────┐     ┌──────────────────┐     ┌──────────────┐
│  PostgreSQL     │     │  Master Key      │     │  Audit Log   │
│  (Metadata)     │     │  (Env Var)        │     │  (Postgres)  │
└─────────────────┘     └──────────────────┘     └──────────────┘

Encryption Flow: 1. File uploaded via API 2. Classification determines if encryption needed 3. Random IV (12 bytes) and salt (16 bytes) generated 4. User-specific key derived from master key + user ID + salt 5. File encrypted with AES-256-GCM 6. Stored as [IV][Salt][EncryptedData] in S3 7. Metadata (IV, salt, classification) stored in database

Document Classification

Level Encryption Retention Use Case
public None No limit Public logos, avatars
internal AES-256-GCM 365 days Receipts, general docs
confidential AES-256-GCM 90 days Invoices, quotes, contracts
restricted AES-256-GCM 30 days Highly sensitive data

Key Storage: - Master key: stored only in the ENCRYPTION_MASTER_KEY environment variable (hex-encoded 32 bytes) - User keys: derived at runtime, never stored - Database: only stores metadata (IV, salt, classification)

Critical: Loss of master key = permanent data loss for all encrypted documents.

Configuration

1. Provision an S3-compatible bucket

Storj, AWS S3, MinIO, etc. Create a bucket (e.g. meisterbill-production). Note the endpoint URL, access key, secret key, and public URL base (if the provider serves public objects via CDN).

2. Generate Encryption Master Key

# Generate a 256-bit (32-byte) key, hex-encoded
openssl rand -hex 32
# Output: <64 hex chars>

3. Configure environment variables

In apps/api/.env (or container env):

S3_ENDPOINT=https://gateway.storjshare.io
S3_BUCKET=meisterbill-production
S3_ACCESS_KEY=your_access_key
S3_SECRET_KEY=your_secret_key
S3_PUBLIC_URL=https://link.storjshare.io/raw/your-bucket-path
# S3_REGION=us-east-1   # optional, defaults to us-east-1
ENCRYPTION_MASTER_KEY=<64 hex chars>

The API reads these via getEnv() at request time. The container runtime (Docker, Kubernetes, hoststack.dev) injects them — there is no Wrangler or Workers binding.

Service

apps/api/src/services/ObjectStorageService.ts — wraps @aws-sdk/client-s3:

  • uploadObject(key, body, contentType, classification, userId) — encrypts (if needed) and uploads
  • getObject(key, userId) — fetches + decrypts
  • deleteObject(key) — removes from bucket
  • headObject(key) — metadata check

The SecureDocumentService (apps/api/src/services/SecureDocumentService.ts) layers classification + audit logging on top.

Audit Logging

Every upload/download/delete emits an event through the event bus. The AuditSubscriber (apps/api/src/subscribers/AuditSubscriber.ts) writes a row to the audit table in Postgres with:

  • event_type — create / update / delete
  • entity — document
  • entity_id — document UUID
  • actor_id — user UUID
  • metadata — IP, user agent, classification

See Audit Logging Setup for the audit subsystem.

Backup

The two stateful components are: 1. Postgres — back up via pg_dump or hoststack.dev's managed backup. Test a restore before relying on it. 2. S3 bucket — use the provider's bucket-level backup (Storj: encrypted-at-rest by default; AWS: versioning + cross-region replication).

Encrypted objects are safe to back up — the master key lives separately in the container's secret store. Without ENCRYPTION_MASTER_KEY, backups are unreadable.

Troubleshooting

"S3 storage not configured" error: - All of S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY, S3_SECRET_KEY must be set - Restart the container after changing env vars

Upload succeeds but download returns garbage: - Master key changed since upload — ENCRYPTION_MASTER_KEY must be stable across restarts - Wrong user context — decryption derives the user-specific key from userId in the request

Audit log row missing: - Check [AuditSubscriber] log lines for errors - Confirm the audit table exists (psql "$DATABASE_URL" -f database/schema.sql if not)