Signals
Environment Drift

Cloud environments diverge from intended patterns, weakening segregation and control.

Divergence between environments that were designed to mirror each other. Testing in staging no longer predicts behavior in production.

How it starts

A production hotfix is applied directly and never backported to staging. A developer gets elevated permissions in a non-production account to debug an issue and the permissions are never revoked. Infrastructure-as-code definitions fall behind the actual state of deployed resources because manual changes are faster. A new service is deployed to production through a side channel because the standard pipeline is too slow for the deadline.

What it looks like

Symptoms that indicate environment drift is active.

  • Code that passes all tests in staging fails in production due to configuration differences.
  • Permission models differ between environments, creating security gaps that only exist in production.
  • Infrastructure audits reveal resources in production that have no corresponding definition in code.
  • Teams cannot reliably recreate production issues in lower environments for diagnosis.
  • Deployment procedures include environment-specific manual steps maintained in tribal knowledge.

Why it matters

Environment drift undermines the fundamental purpose of having environments: to reduce risk by validating changes before they reach production. When environments diverge, the testing process provides false confidence. It also creates security exposure, permission boundaries that are strict in staging but loose in production are invisible until an audit or an incident reveals them.

How we address it

We enforce a single promotion path: code, configuration, and infrastructure definitions follow the same route from commit to production. Environments are defined declaratively and reconciled continuously, drift is detected and flagged, not tolerated. Account boundaries are explicit, and permissions are scoped to the minimum necessary at each stage. The goal is that production is boring because it is the predictable result of a well-understood process.

Where we've seen this

We addressed environment drift at Greenflash, where production hotfixes had created a growing gap between staging and production that undermined the entire promotion process. The single-promotion-path approach in our Infrastructure mandate was refined through that engagement.