Nadella Calls for an 'Emergency Brake' on AI Models: Assume Compromise, Let Humans Hit Stop

What happened

According to TechCrunch's Oct 10 report, Nadella wrote in the post that it is time to "step back and assess the trust architecture" of AI. He argued that "Super Intelligence" — the Trump administration's preferred term for AI — cannot be treated as a set of nested black boxes whose recommendations, answers and actions are simply accepted or rejected.

Nadella's specific proposals include: separating the model from the harness that orchestrates its work; externalizing controls and safeguards outside the model; documenting "every meaningful model action" with "tamper-proof, human-readable evidence"; and ensuring an authorized person can always "pause or shut down a model mid-task." TechCrunch noted the comments come as leading AI companies acknowledge a growing number of incidents in which they seemed to lose control of their models, and after Anthropic CEO Dario Amodei published a plan for more cautious AI development.

Key facts

1. When: Saturday, Oct 10, in a lengthy post on X; first reported by TechCrunch at 2:47 PM PDT the same day
2. Core proposal: Assume a model is compromised and contain it from the start; an authorized person must always be able to pause or shut down a model mid-task
3. Architecture: Separate model from harness; externalize controls and safeguards; tamper-proof, human-readable evidence for every meaningful action
4. Wording: Uses "Super Intelligence" throughout — the Trump administration's official term for AI
5. Context: Follows a string of acknowledged model-control incidents at AI labs, and Anthropic CEO Dario Amodei's recent plan for more cautious AI development