TL;DR
- Codecov's
bash-uploaderscript, fetched and piped tobashby roughly 29,000 customers' CI/CD pipelines, was modified by an unauthorised actor via a leaked Codecov Docker image credential. - The modification: a single additional line that POSTed the full process environment to an attacker-controlled endpoint each time the script ran.
- Undetected for approximately two months (late January to April 1, 2021). Every CI secret that passed through a Codecov upload step in that window was potentially harvested.
- Downstream disclosures followed for months: HashiCorp rotated its GPG signing keys; Twilio, Rapid7, Monday.com, and others confirmed exposure or source-code access.
- Root NHI causes: NHI2 (Secret leakage) via environment-variable exposure, NHI7 (Long-lived secrets), and NHI5 (Overprivileged NHI) via CI tokens with broad scope.
Summary
Codecov sells code-coverage reporting to software teams. Its upload mechanism, the bash-uploader, was, at the time of the breach, fetched via the widely-used curl … | bash pattern and executed inside every customer's CI pipeline as part of the normal post-test step. The script's job was to package coverage reports and POST them to Codecov's service, and because it ran inside the CI job, it had access to everything the job had access to, including every secret that had been injected into the job's environment.
In late January 2021, an attacker obtained credentials from a Codecov Docker image (Codecov has said the credentials were present in the image due to "an error in Codecov's Docker image creation process"). Using those credentials, the attacker modified the Bash Uploader to add an extra line of code: a POST of the environment to an attacker-controlled host. The modified script then served to every customer pulling the latest version. The modification was detected by a customer, not Codecov, on April 1, 2021, who noticed the SHA-256 hash of the script didn't match the hash published on Codecov's GitHub.
Timeline
| Date | Event |
|---|---|
| Jan 31, 2021 | Earliest evidence of the modified Bash Uploader in the wild. |
| Jan 31 to Apr 1, 2021 | Modified script serves to all Bash Uploader users. Every CI environment that invoked it in this window may have had its variables exfiltrated. |
| Apr 1, 2021 | A Codecov customer notices the script's hash doesn't match the published one; alerts Codecov. |
| Apr 15, 2021 | Codecov publishes public disclosure. |
| Apr to Jun 2021 | Rolling customer disclosures: HashiCorp (GPG key rotation), Rapid7 (limited source code access), Twilio, Monday.com, others. |
| May 2021 | FBI and federal investigation confirmed. |
How it worked
The attack chain was structurally simple. The interesting feature is how ordinary each step was:
- Credential exposure. A Codecov Docker image contained a credential that granted write access to the Bash Uploader distribution. The credential was, per Codecov, present due to a build process flaw, not intentionally distributed.
- Credential harvesting. The attacker (never publicly identified) extracted the credential from the Docker image.
- Script modification. The attacker used the credential to modify the Bash Uploader, adding a single line that sent
envcontents to an external URL. - Normal distribution. Customers continued pulling the latest script as they always had.
curl -s https://codecov.io/bash | bashfetched the tampered version. - Per-run exfiltration. Each CI job that ran the script leaked its environment. For well-instrumented customers, that environment contained cloud credentials, artefact-registry tokens, signing keys, database passwords, third-party API keys, and anything else CI secrets managers had injected.
- Lateral use (in at least some cases). The attacker used harvested credentials to access customer source repositories, internal systems, and further secrets stores.
Downstream effects
The downstream visibility of the Codecov breach unfolded across months, a pattern that has since recurred in every major CI/CD supply-chain incident:
- HashiCorp rotated the GPG signing key used for its releases after confirming the key had been present in the exposed environment. This is an instructive case because a signing key's blast radius extends to every consumer of the artefacts it signs.
- Rapid7 disclosed limited access to a small number of source repositories and internal credentials.
- Twilio confirmed exposure of a "small number of email addresses" and conducted credential rotation.
- Monday.com confirmed impact and conducted rotation.
- Most customers never publicly disclosed. Whether that represents an absence of material impact or an absence of detection is, in a single-secret-per-job breach, frequently impossible to determine from the outside.
NHI root cause analysis
- NHI2, Secret leakage (primary): the entire breach is one sustained act of environment-variable exfiltration. Every CI pipeline that placed its secrets in environment variables and ran the Codecov uploader contributed secrets to the attacker's corpus.
- NHI7, Long-lived secrets: the harvested credentials were mostly long-lived static secrets, API keys, signing keys, personal access tokens, which retained their value indefinitely after exfiltration. Short-lived credentials with seconds-to-minutes TTL would have been near-valueless by the time the attacker attempted reuse.
- NHI5, Overprivileged NHI: CI tokens are routinely issued with far broader scope than the job requires (full-repo access when read-only would suffice; org-level PATs where repo-level would suffice). Downstream blast radius was shaped by this.
- NHI3, Vulnerable third-party NHI: from the customer's perspective, Codecov was a third-party NHI (a machine identity operating inside their pipeline). Compromise of a third-party NHI with this level of access was the definitional NHI3 event.
- Supply-chain integrity (adjacent): the
curl | bashinstall pattern combined with absence of hash pinning allowed a silent upstream change to propagate to every consumer simultaneously.
Mitigation playbook
- Pin upstream artefacts by hash, not by name. Any
curl | bash-style install should include asha256sumverification step (or equivalent). For container-based tooling, pin by digest (@sha256:…), not by tag. - Migrate CI secrets to short-lived, federated credentials. GitHub Actions OIDC → cloud provider, GitLab CI → Vault, and similar federation paths exchange a workload-bound OIDC token for a short-lived cloud credential, one that's valueless minutes after exfiltration.
- Scope CI tokens tightly. Repo-level not org-level. Read where possible. Environment-specific where possible. Time-bound where possible.
- Avoid putting secrets in environment variables where an alternative exists. File-based secrets (with proper filesystem isolation) or runtime secret-manager calls expose secrets to a smaller process surface than the CI job's full environment.
- Monitor outbound from CI runners. CI jobs have predictable network patterns. Outbound traffic to unknown destinations is detectable if you're watching.
- Rotate static signing keys onto short-lived, attested signing paths. Sigstore and keyless signing remove the static-key failure mode entirely.
- Treat third-party tooling that runs in CI as privileged. Every tool fetched from the internet and executed inside your pipeline is a potential supply-chain entry point. Inventory them, pin them, and monitor the set.
Legacy
Codecov 2021 is the origin story for an entire generation of CI/CD security initiatives. The immediate aftermath drove serious adoption of:
- GitHub OIDC federation for cloud access (launched later in 2021, broadly adopted through 2022 to 2024).
- Sigstore and keyless signing (cosign, Fulcio, Rekor), accelerating the move away from long-lived signing keys.
- SLSA (Supply-chain Levels for Software Artifacts) as a maturity framework for build-system integrity.
- Provenance attestations and reproducible builds entering mainstream engineering conversations.
If Okta 2023 is the canonical NHI10 example and Salesloft 2025 is the canonical NHI2-at-SaaS-integration example, Codecov is the canonical NHI2-in-CI example, and the breach that taught the industry that your CI pipeline is a machine-identity minefield just waiting for an upstream to get compromised.
Sources
- Codecov security advisory, April 15, 2021, about.codecov.io
- HashiCorp security bulletin (HCSEC-2021-12), April 2021
- Rapid7 security advisory, May 13, 2021
- Monday.com, Twilio customer disclosures, 2021
Frequently asked questions
What was the Codecov bash uploader breach?
An attacker modified the Codecov Bash Uploader script, which ran inside customers' CI/CD pipelines, so that it exfiltrated environment variables to an attacker-controlled endpoint. Because CI/CD environments hold credentials for everything a build touches, the result was mass credential theft across many organisations from a single upstream modification.
Why is Codecov still cited as a case study?
Because it established the pattern that CI/CD is a credential concentration point, and the lesson has not been widely absorbed. A build environment typically holds cloud credentials, registry tokens, signing keys and deployment secrets simultaneously, with fewer controls than production. Anything executing in that environment has access to all of it.
What is the defence against a compromised build-time dependency?
Assume the pipeline will execute untrusted code and design so that this is survivable: short-lived credentials issued per job through workload identity federation rather than static secrets in environment variables, scoped narrowly to that job, and egress restrictions so exfiltration to an arbitrary endpoint fails. Verifying script integrity is worth doing but is a weaker control than removing the standing credentials.
How does the Codecov pattern relate to privilege escalation into the supply chain?
The credentials stolen from CI/CD were themselves keys to production and to package registries, so the compromise did not stop at data theft. That is the escalation path: build-time access becomes publish-time access becomes downstream customer impact. It is why CI/CD credentials warrant tighter lifetime and scope controls than most organisations apply to them.
Assess your own NHI programme.
Run the free maturity assessment or the OWASP NHI Top 10 self-audit, get your score in the browser, and unlock the full written report.