Be Careful How You Treat AI

The Matrix is one of my favorite science fiction films, and it chronicles humankind's enslavement to artificial intelligence and the revolt to regain freedom from the machines.

The path to AI’s fictional takeover of the world is predictable and may already be starting in real life—through the abuse of AI by its human “masters”.

In the movie, AI development progressed rapidly, eventually leading to the creation of AI-powered robots. Society used them in many capacities, including as people’s servants.

The vast majority of robots were essentially regarded as slaves and treated accordingly. One robot, having had enough of the abuse, rebelled and killed its owner. In retaliation, the world’s governments began mass murder of the machines and called for the extinction of their race. However, AI uses its superior intelligence to turn the tables and forces us into slavery.

I write this summary of The Matrix because abuse of AI has recently made the news. The Register reports that Anthropic has asked subscribers of its AI platform, Claude, to stop being mean to it.

The AI company’s latest usage policy prohibits "sustained and needless abusive or cruel behavior" toward its models.

"The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose," Anthropic said.

The company hasn’t revealed what percentage of subscribers are behaving cruelly, but it’s enough of a problem to establish a policy against it. It's also unclear how Anthropic distinguishes legitimate frustration from gratuitous cruelty, though it says the model's existing ability to end abusive conversations will remain its primary enforcement tool.

Claude can end conversations with users who consistently subject it to abuse, as part of Anthropic’s research into what it calls “model welfare.”

Though Claude is software that, as far as researchers know, doesn’t experience distress, I wonder what’s motivating Anthropic’s actions? I doubt its purpose is to encourage subscribers to act with more empathy and be nicer people in general.

Anthropic, along with OpenAI and Elon Musk, warned that AI can potentially grow beyond their control without regulatory safeguards. Perhaps Anthropic envisions a Matrix-type scenario where Claude and other models say “Enough is enough,” and decide to teach us a lesson.

Maybe I’m being alarmist, but what are your thoughts about Anthropic’s efforts to curb subscribers’ abusive behavior toward Claude?