CI/CD Pipelines

The opinion

I have worked in a few CI systems. The CI system matters less than where the logic lives.

Keep the logic in scripts, and the YAML thin

stages: [build, test, scan, deploy]

build:
  stage: build
  script: ["./ci/build.sh"]

test:
  stage: test
  script: ["./ci/test.sh"]

deploy:staging:
  stage: deploy
  script: ["./ci/deploy.sh staging"]
  environment: staging

Every one of those scripts runs on a laptop. "The pipeline is broken" becomes normal debugging, not push-and-wait guessing.

Give each script the same four steps:

1. Authenticate: fail fast if credentials are missing.
2. Execute: do the work.
3. Validate: check the end state, not just the exit code.
4. Roll back: a defined path when validation fails.

Step three is the one most scripts skip. A deploy that exits 0 because the API accepted the request has verified nothing.

The minimum viable pipeline

Protect the main branch. A pipeline you can bypass gates nothing.

Fifteen minutes

Past about fifteen minutes, a pipeline stops giving feedback and starts interrupting. Developers switch tasks, and coming back costs more than the minutes.

• Run independent stages in parallel.
• Cache dependencies, and verify the cache is hit.
• Run slow suites on a schedule, not on every commit.
• Track duration over time. It creeps.

Every failure needs a class

A pipeline that fails for unrelated reasons teaches people to re-run on red. Then it stops carrying information. Classify every failure:

• A real defect: the pipeline did its job.
• Environment or config drift: fix the environment, not the test.
• Flake: quarantine now, then fix or delete it.
• Infrastructure: runner capacity, network, registry.

Count failures by class for two weeks. Most teams find most failures are not defects.

Anti-patterns

• Jobs configured through a web UI. Not versioned, and gone if the server is.
• Copy-pasted pipelines across repos. Twenty slightly different deploys, and no way to fix them together.
• Plain-text pipeline credentials. Use the secret store and short-lived credentials.
• E2E suites as the main gate. See the strategy page.
• Retry until green. Then the pipeline is no longer a safety mechanism.

With Claude

Pipeline work suits AI well. The inputs and failures are text, and the feedback is fast.

Triaging a build failure

Classify first, explain second. Asking "why did this fail" invites a plausible story.

Here is a failing build log.

1. Classify the failure: real defect / config drift / flake /
   infrastructure. State your confidence.
2. Quote the specific log lines that support that classification.
3. Give the most likely cause and the smallest check that would
   confirm or eliminate it.

If the log doesn't contain enough to classify it, say so and tell me
what to capture on the next run instead of guessing.

The last instruction is the one to copy. Without it you get a confident answer built on thin evidence.

Porting between CI systems

Port this CI config from the old system to the new one.

Rules:
  - move all shell logic into ./ci/*.sh scripts, keep the YAML thin
  - preserve stage ordering and the existing branch conditions
  - map repository variables to the new system's variables and list them
  - flag anything with no direct equivalent rather than approximating

Output: the YAML, the scripts, then a table of everything I need to
configure manually before the first run.

Writing the validation step

Give the model the deploy script and ask for post-conditions: what should be true afterward, and how to check it.