Containers & ECS

The sealed package

Most container problems I have debugged in production were not container problems. The app reached out at startup and got a different answer than it did in test.

A container that never calls the cloud API at startup cannot fail on a throttled API, a missing boot-time permission, or a slow regional endpoint.

Configuration comes in

The task definition resolves secrets, not the app's startup code.

{
  "name": "api",
  "image": "…/api@sha256:9f2c…",
  "environment": [
    { "name": "LOG_LEVEL", "value": "info" }
  ],
  "secrets": [
    { "name": "DB_PASSWORD",
      "valueFrom": "arn:aws:secretsmanager:…:secret:prod/api/db" }
  ]
}

The app sees DB_PASSWORD as a plain environment variable. It does not know Secrets Manager exists.

• Inject secrets through the task definition's secrets block.
• Keep KMS out of application code.
• Scope the task role to what the app needs. Usually very little.
• Pin images by digest, not latest.

• No SDK calls for config at startup: no SSM, no KMS, no STS.
• No per-environment config baked into the image.

If it needs cloud credentials to boot, it is not portable

Ask if a developer can run the service on a laptop with no AWS credentials, using a .env file. If not, the app has a hidden dependency on its runtime. That causes bugs that only reproduce in one environment, the most expensive kind to chase.

Image size is a delivery metric

Multi-gigabyte images are usually accidental. Every pipeline run pushes them, and every scale event pulls them.

FROM node:22-slim AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci                      # dev deps live and die here
COPY . .
RUN npm run build

FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev && npm cache clean --force
COPY --from=build /app/dist ./dist
USER node                      # never run as root
CMD ["node", "dist/server.js"]

Put dependency layers before source. A code change should not bust the dependency cache. That is often worth more than the size cut.

Blue/green, and rollback you have tested

Blue/green keeps the previous version running, so rollback is a traffic shift, not a pipeline run. The part teams skip is the health check. /health returning 200 because the web server started tells you almost nothing.

• Make the health check verify the dependencies the service needs.
• Set a deregistration delay for in-flight requests.
• Test the rollback on purpose. An untested rollback is a hypothesis.

Blue/green chooses which version serves. Feature flags choose which users see new behavior.

Anti-patterns

• Running as root. One line to fix. It is the first thing a scanner flags.
• latest in a task definition. You cannot say what is running.
• Secrets as plain environment values. Use secrets. Plaintext ends up in revision history.
• One image per environment. You tested a different artifact than you shipped.
• Vertical scaling as an architecture fix. Cap the instance size. That exposes the real problem.

With Claude

Auditing a Dockerfile

Name the standard. "Is this good?" gets generic advice.

Review this Dockerfile as a container security and build-performance
audit. Structure the output as a table: finding, severity, fix.

Check specifically:
  - runs as non-root
  - multi-stage, with no build toolchain in the final image
  - layer order maximizes cache reuse across code-only changes
  - no secrets or credentials in any layer (including intermediates)
  - base image is pinned and currently supported

Ignore style preferences. Only report things with a concrete impact.

Finding startup-time cloud calls

Search this service for configuration being fetched at startup rather
than injected: SSM parameter reads, Secrets Manager calls, KMS
decrypts, STS assume-role, any SDK client constructed at import time.

For each: file and line, what it fetches, and what it would take to
move it into the task definition instead.

I want to know whether this service can boot with no AWS credentials.
Answer that question explicitly at the end.

Why a task won't start

Give the model the stopped-task reason, the task definition, and the log tail together. The answer is usually in how they interact.