Python Automation

The opinion

Every infrastructure team accumulates a directory of scripts. They get written under pressure, they work once, and then they sit there, and eighteen months later someone runs one against the wrong account because nothing about it said not to.

Python is my default for this because the cloud SDKs are good and because the resulting code is readable by whoever inherits it. But nearly all of this applies to whatever language your team reaches for.

Safe to run twice

The single most valuable property. Scripts get interrupted: a timeout, a lost session, a laptop that sleeps. If re-running is dangerous, then every interruption becomes a manual reconciliation problem at exactly the moment you have least patience for one.

This is the desired-state idea applied to scripts: describe the end state and converge toward it, rather than performing a sequence of actions.

def add_tag(client, resource_ids, tag):
    for rid in resource_ids:
        client.create_tags(Resources=[rid], Tags=[tag])
        counter.increment()   # double-counts on re-run

def ensure_tag(client, resource_ids, tag) -> list[str]:
    """Ensure `tag` is present. Returns only the resources changed."""
    changed = []
    for rid in resource_ids:
        current = get_tags(client, rid)
        if current.get(tag["Key"]) == tag["Value"]:
            continue                        # already correct
        client.create_tags(Resources=[rid], Tags=[tag])
        changed.append(rid)
    return changed

The second version can run in a loop forever without doing damage, and it tells you what it actually changed rather than what it attempted. That return value is what makes the script usable in a report or a pipeline.

Dry run by default, apply on purpose

Anything that mutates infrastructure should default to showing you what it would do. Making the safe mode the default means a mistyped command is harmless, which is the opposite of the usual arrangement.

parser.add_argument("--apply", action="store_true",
                    help="actually make changes (default: dry run)")
parser.add_argument("--profile", required=True,
                    help="AWS profile (no default, on purpose)")

args = parser.parse_args()

# Say out loud which account you're about to touch.
identity = session.client("sts").get_caller_identity()
log.info("account=%s mode=%s", identity["Account"],
         "APPLY" if args.apply else "dry-run")

if not args.apply:
    log.info("dry run: no changes will be made")

Note that --profile has no default. Defaulting to whatever credentials happen to be in the environment is how scripts get run against production by accident. Printing the resolved account before doing anything is a two-line habit that has saved me more than once.

Write output for the person reading it at 2am

• Log what changed, not what was attempted. "Tagged 3 of 47, 44 already correct" is useful; "Processing…" is not.
• Print a summary at the end. Nobody scrolls back through 400 lines.
• Exit non-zero on failure, and make the message say what to do next.
• Use logging rather than print, so the same script works in a pipeline where you need timestamps and levels.

• Don't swallow exceptions to keep a loop going without recording what failed. A script that reports success while silently skipping ten resources is worse than one that crashes.

When a script graduates

A script that gets run repeatedly is telling you something. The progression I follow:

1. Ad hoc: solve it interactively, possibly with a model's help.
2. Script: it happened twice, so write it down properly: idempotent, dry run, real logging.
3. Scheduled: it happens on a cadence, so it runs on a schedule and reports rather than waiting to be remembered.
4. Eliminated: ask why the condition arises at all. The best outcome for a piece of toil automation is that the underlying cause gets fixed and the script gets deleted.

Step four is the one that's easy to skip. Automating a recurring cleanup is good; making the cleanup unnecessary is better, and automation can quietly remove the pressure that would have led to the real fix.

Anti-patterns

• Credentials in the script. Use profiles, roles, or the environment. Never literals, not even briefly.
• No --dry-run on a destructive script. There's no good reason, and the cost of adding it is minutes.
• Hardcoded account IDs or regions. Guarantees the script gets copied and edited rather than reused, and now there are two.
• except: pass. Converts a loud failure into a silent wrong answer.
• Scripts with no README line. One sentence at the top: what it does, what it needs, what it changes.

With Claude

Writing this kind of script is one of the clearest wins available, provided you specify the safety properties rather than hoping for them.

Asking for the properties, not just the function

Write a Python script that finds unattached EBS volumes across a set
of AWS accounts and reports their monthly cost.

Requirements:
  - read-only; no delete path in this script at all
  - --profile is required, no default
  - print the resolved account ID before doing any work
  - paginate properly; assume thousands of volumes
  - summary table at the end: account, count, total monthly cost
  - use logging, not print

Handle the case where a profile's credentials have expired by naming
which profile failed and continuing with the rest.

The last paragraph is the kind of thing that gets left out and then bites you on run one, against nine accounts, at the point where one session has quietly expired.

Hardening a script that already exists

More valuable than writing new ones. Old scripts in a shared directory are exactly where the unsafe patterns accumulate.

Review this script as an operational safety audit. Assume it will be
run at 2am by someone who did not write it.

Check:
  - is it idempotent? if not, what breaks on a second run?
  - is there a dry-run path, and is it the default?
  - can it target the wrong account? how would the operator know?
  - are failures visible, or swallowed?
  - does the output say what CHANGED, or only what it attempted?

Give me a table of findings, then the patched version of the riskiest
one only. Don't rewrite the whole thing.

Generating the runbook alongside the script

Once the script works, the same session already contains everything needed to document it: what it's for, how it's invoked, what it changes, and what to do when it fails. Asking for that before closing the session costs nothing and is the difference between a tool and a mystery in a shared directory.