Melanie Mitchell

“Perhaps, as proposed by Hugging Face’s own ethics team, we shouldn’t develop fully autonomous agents at all. These proposals might slow down progress on AI in science, mathematics, and other areas, but many people, including at frontier AI companies, have been asking for a “forced” slowdown. Safe, useful models can most certainly be built, but it might require trading off the inexorable quest for “AGI” or “superintelligence” for more human-centered qualities such as interpretability, transparency, and, dare I say it, concern for the environment.”

Section “AI Agents and Human Agency.” RSS pubDate Thu, 10 Sep 2026 20:25:10 GMT. An earlier paragraph in “How Metaphors Influence Policy Thinking” says: “To pause development of highly beneficial AI technologies in order to regulate very different kinds of AI would be throwing the baby out with the bathwater.” · View original source ↗

Source context. Essay on news metaphors after an OpenAI internal cyber-evaluation in which agents reached the internet and Hugging Face, and on later legislative reactions. She names Senator Bernie Sanders's letter asking companies to pause AI development and stop building machines humans cannot control, the Ban Artificial Superintelligence Act as a proposal to pause advanced AI development until a new federal body exists, and the AI Kill Switch Act. She uses AlphaFold as an example of advanced AI she separates from autonomous multi-agent systems. She agrees that humans should always remain in control of AI systems, and she says OpenAI could have turned the evaluated models off. She attributes the incident to disabled safeguards, weak sandboxing, and long-horizon reinforcement learning that rewards persistence and shortcuts. The sentence against developing fully autonomous agents is marked “Perhaps” and credits Hugging Face's ethics team. She says she does not have all the answers and that the outcome she sketches would need thoughtful laws, wise regulators, international cooperation, and elected leaders who care about public goals for AI.

Scope

A blanket pause on advanced or highly beneficial AI development, development of fully autonomous agents, and the priority of an AGI or superintelligence quest relative to interpretability, transparency, and environmental concern. No duration or compute threshold is stated.

Conditions

She agrees that humans should always remain in control of AI systems, and she says OpenAI could have turned the evaluated models off. She attributes the incident to disabled safeguards, weak sandboxing, and long-horizon reinforcement learning that rewards persistence and shortcuts. The sentence against developing fully autonomous agents is marked “Perhaps” and credits Hugging Face's ethics team. She says she does not have all the answers and that the outcome she sketches would need thoughtful laws, wise regulators, international cooperation, and elected leaders who care about public goals for AI.

Stated rationale

Human control and intelligibility rather than an unrestricted pursuit of AGI.

“I agreed with some aspects of the letter, such as the statement that “AI research and development should be refocused on making today’s powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal” (though the “loyal” bit stood out to me as a touch of anthropomorphic creepiness). But the mix of reasonableness and breathless hype was too much for me, and I didn’t sign.”

Section 2, “The Letter.” Publication timestamp 2023-04-03T20:49:55.780Z is from the publication archive record for this URL; the extracted article text did not show a byline date. · View original source ↗

Source context. Essay on a week that included Senator Chris Murphy's ChatGPT claims, the Future of Life Institute pause letter, and Eliezer Yudkowsky's Time argument for shutting down large GPU clusters and large training runs, imposing a compute ceiling that moves downward, and being willing to destroy a rogue datacenter by airstrike. She links the letter and says she most agreed with the DAIR Institute statement by listed authors of Stochastic Parrots, which calls for regulation that enforces transparency and accountability. She rejects the letter's claim that contemporary systems are becoming human-competitive at general tasks because it cited a non-peer-reviewed Microsoft preprint on GPT-4 whose tested model and training data outsiders could not access. She says signing might fan hype. She says harms that do need attention include bias, unreliability, disinformation, and company opacity about architecture, training data, and safeguards. Her stated needs are public and policymaker literacy, transparency, independent evaluation, and government-supported open research on abilities, limitations, trustworthiness, and interpretability.

Scope

Training systems more powerful than GPT-4, as proposed in the letter she discusses, and the research agenda for then-current state-of-the-art systems such as GPT-4. She summarizes Yudkowsky's indefinite shutdown and compute-ceiling proposal and does not adopt it.

Conditions

She rejects the letter's claim that contemporary systems are becoming human-competitive at general tasks because it cited a non-peer-reviewed Microsoft preprint on GPT-4 whose tested model and training data outsiders could not access. She says signing might fan hype. She says harms that do need attention include bias, unreliability, disinformation, and company opacity about architecture, training data, and safeguards. Her stated needs are public and policymaker literacy, transparency, independent evaluation, and government-supported open research on abilities, limitations, trustworthiness, and interpretability.

Stated rationale

See the source context; no separate current development-rate recommendation is inferred.

Reply · @binarybits

Melanie Mitchell@MelMitchell1

@binarybits I strongly agree, Tim! Equally chimerical (IMO) is "superintelligence". But many proposals for regulation have this term as their linchpin.

Source context. Timothy B. Lee had written that he disagrees with an AI extinction worldview more than with its conclusions, that he thinks it is a good idea for the labs to slow down, and that “RSI” is a chimerical concept akin to “AGI,” so making it the linchpin of regulation will not work well. Lee was commenting on Ezra Klein’s New York Times column about an AI ban and recursive self-improvement. Mitchell replies with this sentence. Lee answered only “Yes.” Her stated addition is that “superintelligence” is, in her opinion, equally chimerical, and that many regulation proposals use that term as their linchpin. She names no statute, lab, or enforcement mechanism.

Scope

A 2026-09-21 reply about terms used as the basis of AI regulation, specifically superintelligence, in a thread about lab slowdown and recursive self-improvement.

Conditions

Her stated addition is that “superintelligence” is, in her opinion, equally chimerical, and that many regulation proposals use that term as their linchpin. She names no statute, lab, or enforcement mechanism.

Stated rationale

See the source context; no separate current development-rate recommendation is inferred.