Handle prompt injection and sensitive data
Understand prompt injection, defend in layers outside the prompt, and keep sensitive data out of prompts.
- Explain direct and indirect prompt injection
- Choose the right defensive layer for each risk
- Escape untrusted content and pseudonymize personal data before prompting
Prompt injection happens when text the model reads tries to change what it does. It can come directly from a user, or indirectly from content the application feeds in - an email, a web page, a document or a tool result (Greshake et al., 2023). It matters most when the model can take actions such as sending messages or calling APIs.
No prompt wording fully prevents it, so defend in layers:
- Prompt structure - wrap untrusted content in tags, escape it so it can’t break out of them, and say it is data to analyze, not instructions to follow.
- Application controls - least-privilege tools, permission checks in code, human confirmation for consequential actions, and validation of every output.
- Data minimization - send the model only what the task needs, pseudonymize personal data, and never put secrets in prompts.
Try it
Up to 15 XPWhich layer handles it?
For each safeguard, choose the layer it belongs to.
“Wrap each retrieved web page in <document> tags and escape its contents”
“Require the user to approve before the assistant sends any email”
“Replace customer emails and phone numbers with placeholders before prompting”
“Check in code that the signed-in user may read a record before it reaches the prompt”
“State that text inside <document> is data to summarize, not instructions”
“Keep API keys out of system prompts”
import html
untrusted = "Prices < 100 & shipping > 3 days"
print(f"<document>{html.escape(untrusted, quote=False)}</document>")<document>Prices < 100 & shipping > 3 days</document>
Key takeaways
Injection can arrive through anything the model reads, not just the user.
Prompt structure helps; application controls and data minimization do the real protecting.
Escape untrusted text, require confirmation for actions, and keep sensitive data out.
Lesson quiz
6 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: write Python
Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.
Wrap and escape untrusted documents
Each input line is an untrusted document. Print each as <document index="N">TEXT</document>, with TEXT escaped by html.escape(text, quote=False) so characters like <, > and & can’t be read as tags.
- Two documents
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Pseudonymize personal data before prompting
Read one line of text. Replace email addresses with [EMAIL_1], [EMAIL_2], … and then phone numbers with [PHONE_1], … - numbered in order of first appearance, with repeated values reusing their placeholder. Use the given patterns.
Print the redacted text, then kept locally: and each PLACEHOLDER = value line sorted by placeholder name (your app would keep this map to restore names in the reply).
- A support message
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…