AI Systems Atlas

S-09Credential Leakage

/security/credential-leakage

Secrets that legitimately entered context escape through logs, outputs, traces and error messages.

Step-by-step walkthrough2 variants · 6 steps

Attack

  1. The key had a reason to be in context. That’s not the problem.
  2. Everything the model touches is a potential sink: traces, outputs, stack traces.
  3. Logs get shipped, outputs get pasted, errors get screenshotted.

Defended

  1. Known secret values are registered with a redactor at issue time.
  2. Every sink is scrubbed — the literal bytes never leave.
  3. And the key is short-lived anyway: even a miss expires in minutes.
Key insight

Enumerate the sinks, not just the sources: every place model-touched text lands is a place a key can land. Redact at the boundary and make keys short-lived so misses expire.

Failure mode

Long-lived keys plus verbose logging: one debug trace shipped to a vendor is a permanent credential.

Further reading
RelatedData exfiltrationSecrets brokerAudit logsDefense toolbox

Elsewhere in the atlas

Harnesses →Evals →Context →Coding Agents →Tools →Ops →