MANAGE-4 — Manage 4: Post-Deployment Monitoring and Improvement
16 requirements in the Manage 4: Post-Deployment Monitoring and Improvement function
MANAGE 4.1Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and
MG-4.1-001Collaborate with external researchers, industry experts, and community representatives to maintain
MG-4.1-002Establish, maintain, and evaluate effectiveness of organizational processes and procedures for
MG-4.1-003Evaluate the use of sentiment analysis to gauge user sentiment regarding GAI content performance
MG-4.1-004Implement active learning techniques to identify instances where the model fails or produces
MG-4.1-005Share transparency reports with internal and external stakeholders that detail steps taken to
MG-4.1-006Track dataset modifications for provenance by monitoring data deletions, rectification requests
MG-4.1-007Verify that AI Actors responsible for monitoring reported issues can effectively evaluate GAI
MANAGE 4.2Measurable activities for continual improvements are integrated into AI system updates and include
MG-4.2-001Conduct regular monitoring of GAI systems and publish reports detailing the performance, feedback
MG-4.2-002Practice and follow incident response plans for addressing the generation of inappropriate or
MG-4.2-003Use visualizations or other methods to represent GAI model behavior to ease non-technical
MANAGE 4.3Incidents and errors are communicated to relevant AI Actors, including affected communities
MG-4.3-001Conduct after-action assessments for GAI system incidents to verify incident response and recovery
MG-4.3-002Establish and maintain policies and procedures to record and track GAI system reported errors
MG-4.3-003Report GAI incidents in compliance with legal and regulatory requirements (e.g., HIPAA breach