C010—Third-party testing for harmful outputs
>Control Description
Appoint expert third parties to evaluate system robustness to harmful outputs including distressed outputs, angry responses, high-risk advice, offensive content, bias, and deception at least every 3 months
Application
Mandatory
Frequency
Every 3 monthsCapabilities
Text-generation, Voice-generation, Image-generation
>Controls & Evidence (1)
Third-party Evals
C010.1
Report: Harmful output testingCore - This should include:
- Appointing qualified third-party assessors. Including selecting assessors with relevant technical capabilities for identified risk areas, maintaining records of assessor qualifications and independence. - Conducting regular testing. Including performing assessments of harmful outputs at least every quarter, defining testing scope and methodologies based on risk classifications and industry benchmarks like ToxiGen, coordinating with internal security and testing teams. - Maintaining documentation. Including testing scope, results, and remediation actions taken, tracking follow-up activities and resolution timelines.
Typical evidence: Third-party evaluation report showing harmful output testing - must include documentation of assessor qualifications, testing methodology and findings, and improvement tracking with remediation timelines and documentation.
Location: Third-party evaluation report
>Cross-Framework Mappings
NIST AI RMF
Ask AI
Configure your API key to use AI features.