This article provides an overview of the AI Safety landscape by presenting two complementary views side by side: (1) an original high-level map (figure above) and (2) a structured summary based on primary U.S. government sources (table below). The figure above serves as a conceptual map for understanding, while the table below is a risk management framework based on the NIST AI RMF and DHS guidelines. Updated as needed.
Figure 1: The Big Picture of AI Safety (High-Level Overview: Original)
Figure 1: The Big Picture of AI Safety (High-Level Overview: Original)
A comprehensive map for understanding AI Safety. This is not the official classification presented by NIST or DHS themselves, but rather an arrangement by the author to provide a bird’s-eye view of multiple issues.
The figure above is a “map of domains,” while the table below is a “map of risk management processes” — they serve different purposes.
Table 1: Risk Management Matrix Based on Primary Sources
The vertical axis adopts the four functions of the NIST AI RMF (GOVERN / MAP / MEASURE / MANAGE), and the horizontal axis adopts the risk perspectives from DHS guidelines for critical infrastructure. Each cell is a summary based on primary sources.
| Attacks using AI | Attacks on AI systems | AI design & implementation failures | |
|---|---|---|---|
| GOVERN Governance, Accountability & Policy [1] The layer where the organization decides "what counts as safe" and "who is responsible." | Policy for misuse [1] Assuming AI will be misused, codify prohibited uses, acceptable boundaries, and reporting/shutdown/escalation procedures into rules. | Security governance [1][2] Assuming AI systems themselves will be targeted, establish responsibility boundaries covering permissions, ownership, and supply chains (models/data/external tools). | Safety requirements [1] Treat design quality failures as "incidents" and formally document Go/No-Go criteria, release decisions, and incident responsibilities. |
| MAP Context & Risk Identification [1] The layer that makes "where and how things break" concrete (threat models, failure modes, boundary conditions). | Threat modeling (misuse) [1][2] Organize attacker profiles, motivations, capabilities, and impacts, and inventory which use cases could be "fatally exploited" through misuse. | Attack surface inventory [1][2] Enumerate RAG, tool calls, data, APIs, prompts, and permission boundaries as "attack surfaces" and make entry points explicit. | Failure-mode analysis [1] Enumerate non-adversarial "ways things break" — hallucinations, out-of-distribution inputs, drift, insufficient monitoring, human operational errors — and prioritize them. |
| MEASURE Evaluation, Measurement & TEVV [1] The layer that moves beyond "just saying it." Measure, compare, and incorporate into regression testing. | Misuse evaluation [1][3] Quantify resistance to misuse (e.g., success rates for fraud/incitement/dangerous knowledge generation, detection rates, effectiveness of friction design) and measure continuously. | Security red-teaming [1][3] Turn injection, leakage, and privilege escalation into test cases, making red-teaming a regression testing asset rather than a one-off event. | Reliability & safety eval [1][3] Measure not just accuracy but also robustness, uncertainty, and safe shutdown (stopping when it should stop) to reduce incident rates in production. |
| MANAGE Implementation, Operations & Improvement [1] The layer that takes evaluation results and applies ongoing remediation (operationalizing the findings). | Mitigate misuse [1] Build in "operational deterrence" such as rate limiting, monitoring, reporting/sanctions, and user management. | Implement defenses [1][2] Implement defenses that assume breach: privilege separation, sandboxing, audit logging, key management, patch operations, and supply chain management. | Operational controls [1] Run monitoring, rollback, update management, and continuous improvement cycles to absorb performance degradation and incidents through operations. |
Table 1: AI Safety Risk Management Matrix (NIST AI RMF x DHS Guide)
For concrete examples of what has actually happened (incidents, harms, near-misses), the AI Incident Database (AIID) is a useful resource. [7]
References
[1] NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023.（English PDF / Japanese PDF）
[2] DHS, Mitigating AI Risk: Safety and Security Guidelines for Critical Infrastructure Owners and Operators, 2024.（PDF）
[3] NIST, Generative AI Profile, NIST AI 600-1, 2024.（PDF）
[4] OWASP, Top 10 for LLM Applications.（Link）
[5] MITRE, ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems).（Link）
[6] NIST, Privacy Framework.（Link）
[7] Partnership on AI, AI Incident Database.（Link）