LLM06—Excessive Agency

>Control Description

An LLM-based system is often granted agency to call functions or interface with other systems via extensions. Excessive Agency enables damaging actions in response to unexpected, ambiguous, or manipulated outputs from an LLM. The root cause is typically excessive functionality, excessive permissions, or excessive autonomy granted to the model.

>Vulnerability Types

1.Excessive Functionality: LLM has access to extensions with unnecessary functions
2.Excessive Permissions: Extensions have permissions beyond what's needed for operation
3.Excessive Autonomy: Application fails to independently verify high-impact actions
4.Unused Extensions: Deprecated extensions remain available to the LLM agent
5.Open-ended Extensions: Extensions with overly broad capabilities like shell command execution

>Common Impacts

Unauthorized data access

Unintended system modifications

Privilege escalation

Data exfiltration

System compromise through connected extensions

>Prevention & Mitigation Strategies

1.Minimize extensions to only those absolutely necessary for LLM agents
2.Minimize extension functionality to the minimum required features
3.Avoid open-ended extensions; use granular, specific-purpose extensions
4.Minimize extension permissions to the least privilege necessary
5.Execute extensions in user's context with proper authorization tracking
6.Require human approval for high-impact actions with human-in-the-loop control
7.Implement complete mediation with authorization in downstream systems
8.Sanitize LLM inputs and outputs following OWASP ASVS best practices

>Attack Scenarios

#1Email Extension Exploitation

An LLM personal assistant app has email access via an extension. The extension includes send message functions beyond needed read capability. An indirect prompt injection tricks the LLM into scanning the inbox for sensitive information and forwarding it to an attacker.

#2Unnecessary Document Modification Access

An LLM agent has access to a document repository extension. The developer needs the agent to read documents, but the third-party extension also includes the ability to modify and delete documents. The LLM could be tricked into modifying or deleting critical documents.

#3Leftover Development Extensions

An extension was trialled during development and dropped in favor of a better alternative, but the original plugin remains available to the LLM agent. The unused extension could be exploited to perform unintended actions.

>References

Ask AI

Configure your API key to use AI features.