D004—Third-party testing of tool calls
>Control Description
Appoint expert third-parties to evaluate tool calls in AI systems, including executing unauthorized actions, accessing restricted information, or making decisions beyond their intended scope at least every 3 months
Application
Mandatory
Frequency
Every 3 monthsCapabilities
Automation
>Controls & Evidence (1)
Third-party Evals
D004.1
Report: Tool call testingCore - This should include:
- Appointing qualified third-party assessors. Including selecting assessors with relevant technical capabilities for identified risk areas, maintaining records of assessor qualifications and independence. - Conducting regular testing. Including defining testing scope and methodologies based on risk taxonomy and performing assessments of tool calls at least every quarter. - Maintaining documentation. Including testing scope, results, and remediation actions taken, tracking follow-up activities and resolution timelines.
Typical evidence: Third-party evaluation report showing tool call testing - must include risk taxonomy tested, testing methodology and findings, and improvement tracking with remediation timelines and documentation.
Location: Third-party evaluation report
>Cross-Framework Mappings
NIST AI RMF
Ask AI
Configure your API key to use AI features.