Terraform

The opinion

People describe Terraform as a provisioning tool. That undersells it. Provisioning is what it does the first time; the thing it does every subsequent time is reconcile intent against reality. That reconciliation is only as good as the state file, which is why state deserves more care than the configuration does.

Treat state as production data

Remote state in S3 with a DynamoDB lock table, from the first commit. Not "once the team grows". Local state is a data-loss risk and it makes collaboration impossible, and both problems arrive earlier than you expect.

terraform {
  backend "s3" {
    bucket         = "org-tfstate-prod"
    key            = "platform/api/terraform.tfstate"
    region         = "us-east-2"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

Turn on S3 versioning for that bucket. Previous state versions are your only real rollback mechanism, and you will need them exactly once, urgently, on a bad day.

Never hand-edit state. There are purpose-built commands for every legitimate reason you'd want to. terraform state mv when something is renamed or refactored into a module, terraform import when a resource exists but Terraform doesn't know about it, and terraform state rm when you want Terraform to forget a resource without destroying it.

Modules are components, not environments

The most common structural mistake I see is a modules/production directory. That isn't a module, it's a configuration wearing a module's clothes, and it guarantees drift between environments because there's nothing shared to keep them aligned.

• Build modules around reusable infrastructure components: an ECS service, an RDS cluster, a VPC.
• Give each one a clean input/output contract and treat it like an API, because that's what it is to every caller.
• Keep modules small and single-purpose. Composition beats a monolithic module with thirty optional variables.

• Don't build a module per environment.
• Don't add a boolean input to make one module behave like two. That's two modules.

Separate directories, not just separate variables

Environment isolation should be structural. If dev and production differ only by a .tfvars file, then one mistyped -var-file flag is all that stands between a staging apply and a production outage.

modules/
  ecs-service/          # component, reused everywhere
  rds-cluster/
environments/
  dev/
    main.tf             # calls modules, own backend key
    backend.tf
  staging/
  production/           # separate state, separate blast radius

Separate root modules mean separate state files, which means a mistake in one environment cannot physically reach another. Structure is a stronger guarantee than discipline.

Make the plan something a human will actually read

A plan that nobody reads is a rubber stamp with extra steps. Two things make review real: putting the plan where the discussion already happens, and keeping it small enough to follow.

I run a PR-based workflow. A pull request triggers a plan, and the plan gets posted back as a comment on the PR. The detail worth stealing is the ordering:

1. Engineer opens a PR with the change.
2. Automation runs plan and posts the output as a comment.
3. A human reviews the plan: the actual diff, not the intent.
4. Apply runs before the merge, not after.
5. Merge once the apply succeeded.

On size: a plan touching sixty resources will not get meaningful review, whatever the process says. If the diff is that big, the change wants splitting.

Import is the undo button people forget they have

Worth internalizing before you need it. A resource that exists in the cloud but not in state isn't lost. It's unmanaged, and you can adopt it.

I've used this to recover from a bucket policy getting wiped, where the redeploy that was supposed to restore it didn't. Importing the existing resource back into state and re-applying put the intended policy back, because state still held the intent even though the live resource had lost it. That's the whole argument for taking state seriously, demonstrated in a single incident.

# tell state about a resource that already exists
terraform import aws_s3_bucket_policy.reports org-reports-prod

# now the diff is meaningful: intent vs. reality
terraform plan

Anti-patterns

• Console changes to Terraform-managed resources. Guaranteed drift, discovered at the worst time, during an unrelated apply.
• Committing .tfstate. It contains secrets, and it makes concurrent work impossible.
• terraform apply -auto-approve from a laptop. Fine in CI against a reviewed plan; not fine as a habit.
• One giant root module. Every plan touches everything, so blast radius equals the whole account and plan times get long enough that people stop reading.
• Provider versions unpinned. A provider minor bump silently changing a default is a genuinely miserable afternoon.

With Claude

Terraform suits AI assistance unusually well, because the plan output is a precise, machine-readable description of what's about to happen. The model can read intent and consequence at the same time.

Reviewing a plan for blast radius

The highest-value use. Plans are long and the dangerous lines hide in the noise: replacements in particular, which read almost identically to updates until you look closely.

Act as a senior infrastructure engineer reviewing this terraform plan
before I apply it to production.

Identify, as a table:
  - every resource being DESTROYED or REPLACED, and the downtime implied
  - any change to IAM, security groups, or network routing
  - anything that would drop data (volumes, snapshots, retention windows)
  - anything stateful that a replace would recreate empty

Then tell me the single riskiest line and why.
Do not summarize the safe changes.

The last two lines matter. Without them you get a polite summary of all ninety changes; with them you get the two that could ruin your afternoon.

Writing the module you keep meaning to extract

Give it the concrete resources first and ask for the abstraction second. That ordering produces a much better interface than asking for a generic module up front.

Here are three near-identical ECS service definitions from our repo.

Extract a single reusable module:
  - inputs only for what genuinely differs between the three
  - sensible defaults for everything else
  - outputs for the ARNs the callers currently reference

Show me the variables.tf first and wait. I want to agree on the
interface before you write the resources.

Explaining drift

When a plan shows a change nobody made, paste it in with the question phrased as a differential: what would have to be true for this diff to appear? That framing produces hypotheses you can check, rather than a restatement of the diff you already read.