MEASURE-1.1—Approaches and metrics for measurement of AI risks enumerated during the Map function are selected for implementation starting with the most significant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be measured are properly documented.

>Control Description

Approaches and metrics for measurement of AI risks enumerated during the Map function are selected for implementation starting with the most significant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be measured are properly documented.

>About

The development and utility of trustworthy AI systems depends on reliable measurements and evaluations of underlying technologies and their use. Compared with traditional software systems, AI technologies bring new failure modes, inherent dependence on training data and methods which directly tie to data quality and representativeness. Additionally, AI systems are inherently socio-technical in nature, meaning they are influenced by societal dynamics and human behavior. AI risks – and benefits – can emerge from the interplay of technical aspects combined with societal factors related to how a system is used, its interactions with other AI systems, who operates it, and the social context in which it is deployed. In other words, What should be measured depends on the purpose, audience, and needs of the evaluations.

These two factors influence selection of approaches and metrics for measurement of AI risks enumerated during the Map function. The AI landscape is evolving and so are the methods and metrics for AI measurement. The evolution of metrics is key to maintaining efficacy of the measures.

>Suggested Actions

Establish approaches for detecting, tracking and measuring known risks, errors, incidents or negative impacts.
Identify testing procedures and metrics to demonstrate whether or not the system is fit for purpose and functioning as claimed.
Identify testing procedures and metrics to demonstrate AI system trustworthiness
Define acceptable limits for system performance (e.g. distribution of errors), and include course correction suggestions if/when the system performs beyond acceptable limits.
Define metrics for, and regularly assess, AI actor competency for effective system operation,
Identify transparency metrics to assess whether stakeholders have access to necessary information about system design, development, deployment, use, and evaluation.
Utilize accountability metrics to determine whether AI designers, developers, and deployers maintain clear and transparent lines of responsibility and are open to inquiries.
Document metric selection criteria and include considered but unused metrics.
Monitor AI system external inputs including training data, models developed for other contexts, system components reused from other contexts, and third-party tools and resources.
Report metrics to inform assessments of system generalizability and reliability.
Assess and document pre- vs post-deployment system performance. Include existing and emergent risks.
Document risks or trustworthiness characteristics identified in the Map function that will not be measured, including justification for non- measurement.

>Documentation Guidance

Organizations can document the following

How will the appropriate performance metrics, such as accuracy, of the AI be monitored after the AI is deployed?
What corrective actions has the entity taken to enhance the quality, accuracy, reliability, and representativeness of the data?
Are there recommended data splits or evaluation measures? (e.g., training, development, testing; accuracy/AUC)
Did your organization address usability problems and test whether user interfaces served their intended purposes?
What testing, if any, has the entity conducted on the AI system to identify errors and limitations (i.e. manual vs automated, adversarial and stress testing)?

AI Transparency Resources

GAO-21-519SP - Artificial Intelligence: An Accountability Framework for Federal Agencies & Other Entities.
Artificial Intelligence Ethics Framework For The Intelligence Community.
Datasheets for Datasets.

>References

Sara R. Jordan. “Designing Artificial Intelligence Review Boards: Creating Risk Metrics for Review of AI.” 2019 IEEE International Symposium on Technology and Society (ISTAS), 2019.

IEEE. “IEEE-1012-2016: IEEE Standard for System, Software, and Hardware Verification and Validation.” IEEE Standards Association.

ACM Technology Policy Council. “Statement on Principles for Responsible Algorithmic Systems.” Association for Computing Machinery (ACM), October 26, 2022.

Perez, E., et al. (2022). Discovering Language Model Behaviors with Model-Written Evaluations. arXiv. https://arxiv.org/abs/2212.09251

Ganguli, D., et al. (2022). Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned. arXiv. https://arxiv.org/abs/2209.07858

David Piorkowski, Michael Hind, and John Richards. "Quantitative AI Risk Assessments: Opportunities and Challenges." arXiv preprint, submitted January 11, 2023.

Daniel Schiff, Aladdin Ayesh, Laura Musikanski, and John C. Havens. “IEEE 7010: A New Standard for Assessing the Well-Being Implications of Artificial Intelligence.” 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020.