AWS read-only investigation - Strategy

Use this page whenever you need facts from AWS: during an incident, for an inventory, before a decommission, or to answer "what is running where". Everything here is read-only. If a step would change state, stop and follow Terraform changes via merge request instead.

Set up read-only access once

Use AWS IAM Identity Center (SSO) with a read-only permission set, for example ReadOnlyAccess or a custom viewer set that also denies secret reads.
One named profile per account, with a clear suffix such as -ro.
Never use long-lived access keys for investigation.

# ~/.aws/config
[sso-session corp]
sso_start_url = https://<sso-portal>.awsapps.com/start
sso_region = <sso-region>
sso_registration_scopes = sso:account:access

[profile <account-alias>-ro]
sso_session = corp
sso_account_id = <account-id>
sso_role_name = <ReadOnlyRole>
region = <region>

aws sso login --sso-session corp
aws --profile <account-alias>-ro sts get-caller-identity

Always be explicit

--profile and --region on every command. Never trust the default.
Prefer --query and --output json so output is small and parseable.
Loop regions deliberately; most estates use two or three.

P=<account-alias>-ro
for R in <region-1> <region-2>; do
  echo "== $R"
  aws --profile "$P" --region "$R" ec2 describe-instances \
    --query 'Reservations[].Instances[].{id:InstanceId,state:State.Name,name:Tags[?Key==`Name`]|[0].Value}' \
    --output table
done

Recipes for common questions

What is running in this account?

aws --profile "$P" --region "$R" ecs list-clusters
aws --profile "$P" --region "$R" ecs list-services --cluster <cluster>
aws --profile "$P" --region "$R" rds describe-db-instances \
  --query 'DBInstances[].{id:DBInstanceIdentifier,engine:Engine,class:DBInstanceClass,public:PubliclyAccessible}'
aws --profile "$P" --region "$R" lambda list-functions \
  --query 'Functions[].{name:FunctionName,runtime:Runtime,modified:LastModified}'
aws --profile "$P" --region "$R" elbv2 describe-load-balancers \
  --query 'LoadBalancers[].{name:LoadBalancerName,scheme:Scheme,dns:DNSName}'
aws --profile "$P" s3api list-buckets --query 'Buckets[].Name'

Who changed something recently?

aws --profile "$P" --region "$R" cloudtrail lookup-events \
  --lookup-attributes AttributeKey=ReadOnly,AttributeValue=false \
  --start-time "$(date -u -d '-24 hours' +%FT%TZ)" --max-results 50 \
  --query 'Events[].{t:EventTime,who:Username,what:EventName}' --output table

Is anything about to expire or break?

aws --profile "$P" --region "$R" acm list-certificates \
  --query 'CertificateSummaryList[].{domain:DomainName,status:Status,notAfter:NotAfter}'
aws --profile "$P" health describe-events --region us-east-1 \
  --filter eventStatusCodes=open,upcoming

Is this resource still used?

# Load balancer request count over 14 days
aws --profile "$P" --region "$R" cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB --metric-name RequestCount \
  --dimensions Name=LoadBalancer,Value=app/<lb-name>/<lb-id> \
  --start-time "$(date -u -d '-14 days' +%FT%TZ)" --end-time "$(date -u +%FT%TZ)" \
  --period 86400 --statistics Sum

What is this costing?

aws --profile <payer-ro> ce get-cost-and-usage \
  --time-period Start="$(date -u -d '-30 days' +%F)",End="$(date -u +%F)" \
  --granularity MONTHLY --metrics UnblendedCost \
  --group-by Type=DIMENSION,Key=SERVICE

Work across many accounts

Generate profiles from the organization's account list rather than by hand.
Write results to local files per account, then summarise. Do not paste raw output into chats.
Keep the inventory dated; it goes stale fast.

for P in $(aws configure list-profiles | grep -- '-ro$'); do
  aws --profile "$P" sts get-caller-identity --query Account --output text >/dev/null 2>&1 \
    || { echo "skip $P (no session)"; continue; }
  aws --profile "$P" --region "$R" ecs list-clusters --output json > "inventory/$P-ecs.json"
done

Things a read-only agent must not do

Do not call secretsmanager get-secret-value, ssm get-parameter --with-decryption, or kms decrypt. Listing names is fine; reading values is not.
Do not download S3 objects that may hold customer data. Use list-objects-v2 with --max-items to see shape only.
Do not touch an account marked quarantined, frozen or under legal hold. Ask first.
Do not assume a role with write rights "just to look".

Pitfalls

describe calls are regional. "Nothing found" in one region means nothing.
Global services (IAM, CloudFront, Route 53, Organizations) live in us-east-1 for API purposes.
CloudTrail lookup-events covers 90 days of management events only. Data events need a trail and Athena.
Account IDs are not secrets, but they are internal data. Do not publish them.