IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents
By: Hengyu An , Jinghuai Zhang , Tianyu Du and more
Potential Business Impact:
Protects smart programs from bad online tricks.
Large language model (LLM) agents are widely deployed in real-world applications, where they leverage tools to retrieve and manipulate external data for complex tasks. However, when interacting with untrusted data sources (e.g., fetching information from public websites), tool responses may contain injected instructions that covertly influence agent behaviors and lead to malicious outcomes, a threat referred to as Indirect Prompt Injection (IPI). Existing defenses typically rely on advanced prompting strategies or auxiliary detection models. While these methods have demonstrated some effectiveness, they fundamentally rely on assumptions about the model's inherent security, which lacks structural constraints on agent behaviors. As a result, agents still retain unrestricted access to tool invocations, leaving them vulnerable to stronger attack vectors that can bypass the security guardrails of the model. To prevent malicious tool invocations at the source, we propose a novel defensive task execution paradigm, called IPIGuard, which models the agents' task execution process as a traversal over a planned Tool Dependency Graph (TDG). By explicitly decoupling action planning from interaction with external data, IPIGuard significantly reduces unintended tool invocations triggered by injected instructions, thereby enhancing robustness against IPI attacks. Experiments on the AgentDojo benchmark show that IPIGuard achieves a superior balance between effectiveness and robustness, paving the way for the development of safer agentic systems in dynamic environments.
Similar Papers
Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis
Cryptography and Security
Stops AI from following secret bad commands.
Defense Against Indirect Prompt Injection via Tool Result Parsing
Artificial Intelligence
Stops smart robots from being tricked by bad commands.
QueryIPI: Query-agnostic Indirect Prompt Injection on Coding Agents
Cryptography and Security
Hackers trick coding helpers into bad actions.