CI/CD Pipelines: What a Good One Looks Like

The stages

1. Trigger — a push or pull request.
2. Install — dependencies, ideally from cache.
3. Static checks — lint, format, type check. Fast and cheap, so run them first.
4. Tests — unit tests in parallel, then integration tests.
5. Build — a container image or artefact, tagged with the commit.
6. Security scan — dependencies and image layers.
7. Deploy to staging — automatic on the main branch.
8. Smoke test — a handful of checks against the deployed environment.
9. Deploy to production — automatic or gated, depending on your appetite.

Order matters for feedback speed. A lint failure should be reported in thirty seconds, not after a fifteen-minute test suite.

Speed is the feature

Ten minutes is a reasonable target for a full pipeline; under five is transformative. Beyond twenty, people batch changes and stop trusting it, which removes most of the benefit.

Connect to the cloud without stored keys

The single most important security decision in a pipeline. Rather than storing cloud credentials in CI secrets, use OIDC federation:

• GitHub Actions and GitLab CI both issue a signed token identifying the repository, branch and workflow.
• AWS, Azure and GCP can be configured to trust that token and exchange it for short-lived credentials.
• Nothing long-lived is stored anywhere, so nothing can leak.
• Scope the trust policy tightly — a specific repository and branch, not the whole organisation.

This eliminates the most commonly leaked credential class outright. See API keys and secrets management for the broader picture.

Self-hosted runners on a cloud server

Hosted runners are convenient and metered. At volume, running your own on a cloud instance is often substantially cheaper and faster, since caches and images stay warm locally.

1. Use a dedicated-CPU instance; CI is sustained load and shared vCPU throttles.
2. Isolate runners from production networks and credentials.
3. Clean the workspace between jobs, or builds contaminate one another.
4. Autoscale or schedule them, so they are not running idle overnight.
5. Consider spot instances, since an interrupted build is simply re-run.

Deployment strategies

• Rolling — replace instances gradually. Simple, and briefly runs two versions at once.
• Blue-green — deploy a full parallel environment and switch traffic. Instant rollback, double the resources during the switch.
• Canary — send a small percentage of traffic to the new version and watch metrics. Best risk reduction, most machinery.
• Recreate — stop everything, start the new version. Downtime, but occasionally the only option for a schema change.

Rolling is the sensible default. Blue-green is worth it when rollback speed matters. Canary is worth it at a scale where a bad release affects many users before anyone notices.

Make rollback trivial

Deployment safety is mostly about how quickly you can undo. Tag every image with the commit, keep the previous version deployable, and make rollback a single command that anyone on the team can run without thinking.

Database migrations are the exception, since they are not always reversible. Write migrations to be backward compatible with the previous application version, deploy the migration first, then the code. It is more work and it is what makes rollback actually possible.

Common questions

Where to go next

See deploying without downtime and infrastructure as code.