AI Systems Atlas

S-03Direct Prompt Injection

/security/direct-prompt-injection

The user themselves is the attacker: the prompt tries to override system rules and misuse the agent’s tools.

Key insight

Never give an agent capabilities its least-trusted user shouldn’t hold. If the allowlist doesn’t contain the dangerous action, jailbreaking the model accomplishes nothing.

Failure mode

Relying on the model’s refusal as the only barrier between any user and destructive tools.

Further reading
RelatedIndirect injectionConfused deputy

Elsewhere in the atlas

Harnesses →Evals →Context →Coding Agents →