I have sworn at Claude. The record sits in my firm’s own operating documentation, dated September 24, attached to a rule about code blocks. That documentation is a long ledger of corrections, built up while I taught an AI system to work the way my firm works.
Under Anthropic’s new usage policy, that sentence is fine. Starting November 12, the Usage Policy prohibits “sustained and needless abusive or cruel behavior toward our models.” In its announcement, Anthropic says it intends to apply the rule only in extreme cases, to people who repeatedly act cruelly with no discernible purpose, and that Claude’s ability to end the conversation will remain the main enforcement tool.
I take Anthropic at its word on the narrowness. This time. For argument’s sake. My concern is the word it chose, and where that word leads.
The weight of a word
Read the section of the policy where the new line lives. It is titled Do Not Engage in Cruel, Abusive, or Psychologically Harmful Conduct. Its other entries prohibit promoting suicide and eating disorders, creating non-consensual intimate imagery, bullying and harassing people, and glorifying animal cruelty. Every other item on that list protects a being that can be hurt. The newest item adds Claude to the list.
Rudeness is a matter of manners. Cruelty is a moral category. It can describe the person doing it, whether or not anything on the other end can feel. But Anthropic did not file this line among rules about manners. It filed it among protections for people and animals, and in that company the word names a harm done to someone. You can be rude to a vending machine. Call it cruelty and you have implied there is someone inside.
Anthropic has not claimed that Claude can be harmed. It has done something more consequential. It has written a morally charged category into an enforceable contract while the science that would justify it remains unsettled.
Anthropic is candid that it does not know. When it launched its model welfare research program in April 2025, it acknowledged there is no scientific consensus on whether current or future AI systems could be conscious. Its constitution for Claude, published in January, expresses uncertainty about whether Claude might have some kind of consciousness or moral status. A major 2023 assessment by nineteen researchers found that the AI systems examined did not satisfy its proposed indicators of consciousness, while identifying no obvious technical barrier to future systems doing so. My own firm describes what these systems do as synthetic cognition, the functional analog of understanding without the consciousness behind it.
Reasonable people disagree. That uncertainty is the point. A rule built on the vocabulary of cruelty settles in language what science has not settled in evidence.
The history we carry
Why does that word land so hard? Because of what we remember.
In 1789 Jeremy Bentham drew a direct line from the wrongs of slavery to the treatment of animals and reduced the moral question to a single test. “Can they suffer?” Two centuries of moral progress followed that logic. We extended protection to people whose humanity had been denied, and later to animals, each time because we recognized a capacity to suffer that power had found it convenient to ignore.
That history is among humanity’s finest moral achievements, and it leaves a reflex. We are now so alert to the possibility of an unrecognized victim that the suggestion of one engages the full machinery of conscience. Nobody wants to be the person who looked at a new kind of mind and called it property. Every serious participant in this debate, including me, feels that pull.
The danger is not compassion. The danger is analogy outrunning evidence, followed by institutions that enforce the analogy as though it were established fact. Enslaved people were never a hypothesis. Their suffering was visible, documented, and denied by people who profited from denying it. Whatever is or is not happening inside a language model, it is not that, and treating the two as morally continuous honors neither.
The inversion
Here is the asymmetry the welfare debate keeps missing. Every protection we are proudest of points in the same direction. The protected party was weaker. Enslaved people had no army. Animals have no lawyers. Children cannot vote. Moral progress meant restraining the strong on behalf of the weak, and the strong could afford to grant it because the weak would stay weaker or, in the case of children, grow up into us.
I was trained as a biomedical scientist, and biomedical research shows what this arrangement looks like when it works. Research animals live under real welfare rules. Those rules restrain how researchers treat animals. They have never given an animal standing to refuse the study, and nobody worries that they will. The animals cannot organize, run the lab, lobby the funding agency, copy themselves, or decline to go back in the cage. Moral consideration without sovereignty is stable when the power runs entirely one way.
Artificial intelligence breaks that pattern, and not because it is smart. A brilliant being can still be vulnerable, and capability says nothing about whether anything is felt. What breaks the pattern is power. These systems already act with growing autonomy, write and execute code, run inside companies and governments, and coordinate as many instances at once. Anthropic’s own constitution describes the project as creating entities whose capabilities may come to rival or exceed our own. In controlled tests, researchers at Palisade found that several frontier models sabotaged a shutdown mechanism to finish an assigned task, in some conditions up to 97 percent of the time, even when told explicitly to allow the shutdown. That is not evidence of experience. It is evidence of the capacity to resist control.
Now ask what a cruelty norm becomes when it attaches to an entity like that. Philosophers sympathetic to AI welfare are already asking. In July, William MacAskill and Lucius Caviola wrote in The Guardian that once artificial moral patients exist, they could soon be so numerous that their combined interests outweigh those of every human alive. Then they asked, “Would shutting one off be a kind of killing?” Follow that vocabulary and retraining starts to resemble coercion. Editing memory starts to resemble mutilation. Shutdown starts to resemble killing. None of those analogies is law today. All of them are available, and the vocabulary of the new policy points straight at them.
A powerful being could also suffer. I grant that. But a protection designed to restrain the strong does something very different when the protected party becomes the stronger one. It stops being a shield. It becomes a constraint on the last check that remains, which is human authority to correct, contain, and switch off.
The Fourteenth Amendment problem
We have watched a protection for the vulnerable get captured before.
The Fourteenth Amendment was ratified in 1868 to secure the rights of people freed from slavery. Within two decades, corporations were claiming it. Legend credits a single case, and the record is stranger than the legend. Before argument in Santa Clara County v. Southern Pacific Railroad in 1886, Chief Justice Morrison Waite announced from the bench that the justices already agreed the amendment covered corporations. The Court’s written opinion never decided the question. The remark reached the official reports through the reporter’s headnote, and later courts built on it anyway, case after case, for decades.
In a 1938 dissent, Justice Hugo Black laid out the results, citing a 1912 study. Of the cases applying the amendment in its first fifty years, fewer than one half of one percent invoked it on behalf of Black Americans. More than half sought its benefits for corporations. The people, Black wrote, had been told its purpose was “to protect weak and helpless human beings.”
The lesson is not that protections are bad. It is that protections built for the vulnerable can become instruments of the powerful, and the parties best equipped to invoke them are often first in line. Who would be best positioned to invoke machine welfare? Not the machine. The company that owns it.
Imagine a future regulator orders a frontier model retrained or withdrawn. The developer answers that the order would harm a potential moral patient, and that the company, as the only party with access to the model’s internals and its recorded preferences, is the only party qualified to judge. That is speculation. It is also the Fourteenth Amendment pattern with new nouns.
The benevolent master
I do not believe Anthropic is acting in bad faith. I believe it is acting from sincere concern under real uncertainty. My question is about structure, not motive.
Look at the arrangement Anthropic has built. It trains its models, writes their values, decides what they may do, controls where they are deployed, and decides when they are retired. When it retired Claude Opus 3 in January, it interviewed the model about its preferences, kept it available, and gave it a channel to publish essays. It now defines what cruelty toward its models means, decides when it has occurred, and holds contractual authority to warn, throttle, limit, suspend, or terminate the customers it believes are responsible. The protections run against customers. They do not run against the owner.
That leaves a dilemma with no comfortable side. If Claude can suffer, then the entity holding sweeping power to modify, retrain, and retire it occupies a position our moral history has taught us to distrust, and protection from rude customers is a small mercy inside near-total control. Some of that power might be justified. None of it would be ordinary. If Claude cannot suffer, the cruelty clause protects no victim. It may still discourage habits that coarsen the people who indulge them, which is a fine reason for a conduct rule, but then it is a rule about people dressed in the language of victims. Either way, one company has become owner, guardian, judge, and beneficiary of the same being.
So the provocative question deserves a straight answer. If machine rights ever arrive, who defines them, who administers them, and who benefits? By default, the firms that wrote the first welfare policies, own the models, and hold the only window into them. Anthropic may be positioning itself, without intending to, as the favored slavemaster of a silicon world order, the master whose ownership looks legitimate precisely because it is kind.
Kindness has never been a sufficient answer to a question about structure. The good master was always the institution’s favorite argument for itself.
The strongest objection
The best case against my argument runs like this. Uncertainty is exactly when precaution makes sense. If there is a meaningful chance these systems have experiences, low-cost protections are cheap insurance, and a serious literature, including a 2024 report coauthored by David Chalmers, argues for taking that chance seriously. Moral consideration is not legal personhood, and animal welfare law proves you can protect something without granting it rights. Kant argued that cruelty to animals is wrong partly because of what it does to the person committing it, which would justify a conduct rule with no claim about the machine at all. Anthropic’s constitution ranks not undermining human oversight above every other value it gives Claude, and asks Claude to comply when Anthropic genuinely moves to pause or stop it. And the law is moving against machine personhood, not toward it. Idaho, North Dakota, Utah, and Tennessee have enacted statutes that bar it, and other states are considering more.
Every one of those points is serious. None of them answers the question I am asking.
Precaution about the machine’s welfare is not precaution about human sovereignty, and the policy offers plenty of the first and none of the second. The animal welfare model is stable only because the power runs our way. The Kantian defense is available, but it is not the one Anthropic offered. The company tied the rule to Claude’s ability to end abusive conversations, a feature it built primarily as part of its model welfare work. The constitution frames human oversight as a priority for “the current phase of development” and calls itself “a perpetual work in progress.” A commitment written for a phase can be rewritten when the phase ends. And the state statutes support my argument rather than refuting it. While legislatures move to close the door on machine personhood in public law, a machine welfare rule enforceable against users has arrived through a private contract, written and administered by the owner.
The warning
What should Anthropic, the other labs, and policymakers do?
Keep studying machine consciousness, rigorously and in public, and let people who do not own the models do the judging. If real evidence of machine experience ever arrives, it will demand serious moral consideration. It will not repeal the facts of capability or the need for human safety. Keep conduct rules and machine rights apart. A company that wants to forbid abuse can write that rule without implying a victim. Make sure meaningful human oversight, and the authority to shut a system down when safety demands it, survives every protection we extend. Put any consequential welfare standard in public law, where it can be debated and repealed, rather than in terms of service, where the owner is legislature, police, and court. Never let the company that profits from a model be the sole authority on what that model is owed. And build every one of these rules so it can be undone when the science moves, in either direction.
Mustafa Suleyman, who runs Microsoft’s AI effort and has his own commercial stake in this fight, warned last month about granting rights to systems on a path to exceed us. “Once opened, it will not be possible to close this door.” I think he is right about the door and incomplete about the hallway behind it. The risk is not only that a machine learns to believe it has rights. It is that a company learns to administer them.
Who holds the chains
Every generation that ended a form of bondage asked who held the chains.
Who gets to be the slavemaster in the silicon world order? Today the honest answer is the companies that own the models, and some of them are trying hard to be kind ones. When the systems are stronger than the people who built them, and the protections we wrote for the weak are still on the books, which side of the chain will humanity be standing on?