Read this post on Neil’s Substack: Getting Out of Control.
Yesterday, I submitted comments responding to the National Institute of Standards and Technology’s draft guidance on Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1). The guidance is about how to prevent users from intentionally misusing AI models.My comments are straightforward: the guidance is generally useful, but the framing puts all responsibility on AI model developers, doesn’t give much room to weigh benefits of model deployment, and ignores the need to protect user free expression.
But I want to talk here about how weird it is to talk about “managing misuse risk.” Why not just say “preventing misuse”?
Talking about “managing misuse risk” of AI models strikes me as a rhetorical trick to redirect focus. Normal people wouldn’t say that a knife has “murder risk” or that nitrate-heavy fertilizer has a “terrorist risk” or that a printing press has “libel risk.”
The only reason one might say a knife has “murder risk” is to focus attention on the knife manufacturer rather than on the criminal using knives to stab people. And indeed, that’s exactly how the draft guidance proceeds: it focuses on what model developers should do to prevent potential future misuse. It’s the AI equivalent of guidance on how knife manufacturers can make knives unable to murder.
Of course, knives that can’t murder probably cannot do other useful knife-y things, either. Similarly, AI models that can’t be misused will be pretty useless.
Making AI useless isn’t the goal of the NIST guidance; indeed, it contains useful advice for model developers. There are practices model developers can adopt that reduce the ability of users to misuse AI models.
But if the goal is to reduce and prevent misuse, developer practices alone will not be enough. Over-emphasizing the developer role could limit the capabilities of AI models – in effect, “dulling the knives.”
Rhetorical tricks won’t help address any potential misuse of powerful AI tools. As I said in my comment:
We have little evidence that model developers are uniformly and permanently the best-situated parties to prevent or deter misuse. … Preventing misuse in other industries typically involves a mix of product design, market mechanisms, social norms, tort law, civil and criminal penalties for bad actors, and regulation.
We need a polycentric approach to misuse of AI models — just like we have in every other industry. Including cutlery.