Defend against AI-enabled attacks and unsafe agents

Attackers can use AI to write convincing messages, translate lures, summarize stolen material, adapt scripts and help coordinate tasks. This often scales familiar attacks rather than creating a new kind of vulnerability. In a 2025 case study, OpenAI reported banned accounts that used AI for multilingual phishing text and scripting support; its investigation did not find a novel offensive capability in that case.

Voice and image synthesis can also impersonate a trusted person. CISA advises verifying unexpected requests through a known channel. A familiar voice, polished writing or a video is no longer enough to authorize a payment, reset or data transfer.

An AI agent may read webpages, tickets, emails or repository files and call tools. Those sources are data from someone else, even when they contain sentences that look like commands. Indirect prompt injection puts instructions inside that lower-trust content to redirect the agent. OWASP explains the mechanism.

The damage depends on what the agent is allowed to do. An assistant that only reads a ticket can make a bad summary; one that can read secrets and send messages may leak them. OWASP calls broad tools, broad permissions or unchecked autonomy excessive agency. Other risks include sensitive data disclosure, poisoned source material, unsafe use of generated code and vulnerable model or tool dependencies. These overlap with familiar access-control and supply-chain problems.

A safer design separates the user’s instruction from retrieved content, gives the agent read-only, task-scoped tools by default, applies normal authorization to every tool call, validates outputs before they reach other systems, and requires review for sending, deleting or changing data. Log tool calls and test with adversarial documents. A warning in the prompt alone is not a permission boundary. OWASP’s excessive-agency guidance illustrates the same read-versus-send distinction.

This area changes quickly. Keep model and agent evaluations current, retest after adding tools or changing permissions, and examine both successful attacks and false alarms. NIST’s Cyber AI Profile is still a preliminary draft; use it as a planning aid, not a promise that one architecture is safe.