Development, Staging and Production: Keeping Them Honest
A staging environment that differs from production in the wrong ways gives false confidence, which is worse than no confidence.
The purpose of staging is to make a deployment boring. It only does that if passing there tells you something true about production. Many staging environments differ in precisely the ways that matter and match in the ways that do not.
Differences that are fine
- Scale. One instance instead of six. Behaviour is what you are testing, not capacity.
- Instance size, provided the software is identical.
- Data volume, as long as the schema matches and you test performance separately.
- Redundancy. Single-zone staging is reasonable.
- Uptime. Staging can be off overnight and at weekends.
- Domain and certificates.
Differences that cause incidents
- Different runtime or database versions. The classic source of “it worked in staging”.
- Different operating system or base image.
- Different configuration structure, rather than different values.
- Missing components — no load balancer, no CDN, no cache in staging.
- Different TLS termination, so header handling differs.
- Different feature flag defaults.
- No staging equivalent of a background worker or scheduled job.
The pattern: differences in scale are safe, differences in software or topology are not.
Getting parity without paying for it twice
- Define both environments in the same infrastructure code with different variables. Structure identical, sizes different.
- Use the same container images, promoted between environments rather than rebuilt.
- Pin runtime and database versions identically.
- Include every architectural component in staging, at the smallest available size.
- Schedule staging to shut down outside working hours — about 70 per cent saved.
- Use the cheapest instance sizes; you are validating behaviour, not throughput.
Same shape, smaller size is the principle. A staging load balancer with one small instance behind it costs little and catches real problems.
The data question
Production data in staging is realistic and creates a serious data protection problem: staging usually has weaker access controls, more people with access, and less monitoring.
- Anonymised copy — restore production data with personal fields replaced. Realistic shape, no exposure. Usually the right answer.
- Synthetic data — generated to match the schema. Safest, and can miss edge cases real data contains.
- Subset — a recent slice rather than everything. Cheaper and faster to refresh.
- Raw production data — only with the same controls as production, which usually defeats the purpose.
Whatever you choose, refresh it on a schedule. Staging data from two years ago stops resembling production quickly.
How many environments
| Environment | Purpose | Worth having when |
|---|---|---|
| Local | Fast iteration | Always |
| Preview per branch | Review changes in isolation | Ephemeral environments are cheap to create |
| Staging | Pre-production validation | Almost always |
| Production | Real users | Always |
| Performance | Load testing at scale | When capacity questions are real |
Ephemeral preview environments per pull request are excellent where the stack supports them: created on open, destroyed on merge, so cost is proportional to activity.
Keeping local development close
- Use containers locally with the same base images and versions as production.
- Docker Compose can run the real database engine rather than a lighter substitute.
- Keep configuration in environment variables everywhere, with different values.
- Where a cloud service has no local equivalent, use a shared development instance rather than a different technology.
- Never let local development be the only place a code path is exercised.
Common questions
Can we skip staging and test in production?
With feature flags, canary deploys and strong monitoring, some teams do — and they have invested heavily in the machinery that makes it safe. Without that, staging is much cheaper than the alternative.
How much should staging cost?
Typically 10 to 25 per cent of production, less with scheduled shutdown. If it approaches production cost, it is over-provisioned for its purpose.
Should staging use production credentials for third-party services?
No. Use sandbox or test credentials. A staging test that sends real emails or charges real cards is a memorable incident.
How often should staging data refresh?
Weekly or monthly is common. More often if schema changes are frequent, which is exactly when stale data causes confusing failures.
Where to go next
See CI/CD pipelines and infrastructure as code.