TO: Head of AI Safety, National Institute of Standards and Technology (NIST)
SUBJECT: Synthesizing Expert Knowledge on AI Safety Risks
How can NIST best synthesize a diversity of expert knowledge on AI safety?
NIST plays a crucial role in fostering innovation and mitigating risks associated with AI. Synthesizing diverse knowledge of AI experts demands more than traditional approaches. Status-quo workshop formats are inadequate mechanisms to achieve NIST's mission. Luckily, better approaches are readily available.
-
Traditional: Rely on established NIST workshop processes
-
Novel: Explore lesser-known methods
-
Hybrid: Blend traditional approaches with new techniques
While traditional NIST workshops have some merits, they have major drawbacks:
-
To the extent agendas are predetermined, top-down, and inflexible, they hinder iterative reframing by participants
-
Feedback methods may be burdensome to NIST staff and unengaging to participants
-
Lack of emphasis on creating shared risk models and artifacts hinders shared learning
-
Limited participant input on deliverables misses opportunities for iterative refinement
Novel methods offer promising alternatives:
-
Collaborative argument mapping visually represents the structure of debates, helping identify areas of agreement, disagreement, and gaps.
-
Collaborative Bayesian network building enables experts to jointly model the complex interplay of risk factors and uncertainties.
-
Scenario planning workshops help surface plausible future risks and foster preparedness.
-
Participatory system mapping engages experts in visualizing the dynamics shaping AI safety, leading to shared understanding and identification of intervention points.
-
Prediction markets aggregate diverse opinions and incentivize careful reasoning about AI safety outcomes.
-
Take proactive steps to ensure that existing power structures will not undermine an unbiased exploration of novel methods.
-
Allocate funding and resources in ways that encourage positive-sum thinking to prevent infighting.
-
Bring in outside expert facilitators experienced with these novel collaborative methods.
-
Craft a strategy that accentuates experimentation in the workshop process itself.
-
Form hypotheses about the pros and cons of many methods (traditional, novel, and hybrid). Design and test experiments. Iterate and refine, aiming towards a set of promising hybrid approaches.
-
After some number of iterations, run hybrid workshops. Continue to iterate and refine.
-
Update the organizational structure to reduce chances of workshop strategy lock-in.