GOVERN-6.1—Policies and procedures are in place that address AI risks associated with third-party entities, including risks of infringement of a third party’s intellectual property or other rights.

Risk measurement and management can be complicated by how customers use or integrate third-party data or systems into AI products or services, particularly without sufficient internal governance structures and technical safeguards.

Organizations usually engage multiple third parties for external expertise, data, software packages (both open source and commercial), and software and hardware platforms across the AI lifecycle. This engagement has beneficial uses and can increase complexities of risk management efforts.

Organizational approaches to managing third-party (positive and negative) risk may be tailored to the resources, risk profile, and use case for each system. Organizations can apply governance approaches to third-party AI systems and data as they would for internal resources — including open source software, publicly available data, and commercially available models.

Organizations can document the following

• Did you establish mechanisms that facilitate the AI system’s auditability (e.g. traceability of the development process, the sourcing of training data and the logging of the AI system’s processes, outcomes, positive and negative impact)?
• If a third party created the AI, how will you ensure a level of explainability or interpretability?
• Did you ensure that the AI system can be audited by independent third parties?
• Did you establish a process for third parties (e.g. suppliers, end users, subjects, distributors/vendors or workers) to report potential vulnerabilities, risks or biases in the AI system?
• To what extent does the plan specifically address risks associated with acquisition, procurement of packaged software from vendors, cybersecurity controls, computational infrastructure, data, data science, deployment mechanics, and system failure?

AI Transparency Resources

• GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities.
• Intel.gov: AI Ethics Framework for Intelligence Community - 2020.
• WEF Model AI Governance Framework Assessment 2020.
• WEF Companion to the Model AI Governance Framework- 2020.
• AI policies and initiatives, in Artificial Intelligence in Society, OECD, 2019.
• Assessment List for Trustworthy AI (ALTAI) - The High-Level Expert Group on AI - 2019.

Bd. Governors Fed. Rsrv. Sys., Supervisory Guidance on Model Risk Management, SR Letter 11-7 (Apr. 4, 2011)

“Proposed Interagency Guidance on Third-Party Relationships: Risk Management,” 2021.

Off. Comptroller Currency, Comptroller’s Handbook: Model Risk Management (Aug. 2021).