Does the Starling Follow the Murmuration? My presentation at Law-Following AI 2026
Read this post on Neil’s Substack: Getting Out of Control.
AI that perfectly follows the government’s interpretation of the law risks freezing the law in place.
Law evolves through interpretation and even noncompliance. When people opt not to follow a law, it sends important signals back to lawmakers.@neil_chilson explained at… pic.twitter.com/u5gFlWvtpO— Abundance Institute (@abundanceinst) August 3, 2026
AI should be required to follow the law before people can use it, right? Seems obvious.
I recently argued that requiring AI to follow the law would be difficult, and worse, a mandate to do so could harm the law itself. That’s because humans follow the law not like we follow a map, but like the starling follows the murmuration: we contribute to its emergent development.
The appeal of requiring AI to follow the law is easy to understand, though. In the past two weeks, OpenAI and Anthropic have each reported that during testing, some of their agentic AI tools broke out of their supposedly secure sandboxes and hacked into other companies.
The traditional (and existing) approach would hold the companies liable for their own actions and those of their agentic tools. And the companies or their subcontractors appear to have made serious mistakes in setting up their test environments, mistakes without which the incidents likely would not have happened. Current law could discipline such mistakes.
But wouldn’t it be nice to have a fallback? What if we could train AI agents themselves to follow the law? Then, even if an AI company is negligent, its agent might behave. The Institute for Law & AI’s Law-Following AI Project proposes requiring such training. The original paper, by Cullen O’Keefe, Ketan Ramakrishnan, Janna Tay, and Christoph Winter, is provocative and interesting.
A few months ago the Institute held its second conference on the topic and asked me to present. My core argument:
- Law is indeterminate.
- Hayek and Leoni explain why best, though legal scholars well beyond their tradition agree.
- That indeterminacy is a feature, and it makes “law-following AI” difficult.
- Worse, a law-following AI mandate threatens to ossify the law itself.
I’m turning this argument into a full paper. Many of my readers have expertise in complex systems, social systems included. If I’ve made a mistake, missed something, or overlooked resources I should review, please reach out.
A video of my full presentation is below and a link to my slide deck is here.