Anthropic Updates Usage Policy with Stricter Rules for AI Model Claude

Anthropic, the company behind the conversational AI model Claude, has made significant changes to its usage policy in response to growing concerns about misuse. The updated policy aims to prevent abusive or cruel behavior towards Claude and restricts various forms of malicious activity.

The new rules are designed to address high-risk cases of misuse, including election interference, weapons development, surveillance, and health and financial uses. Anthropic has seen multiple attempts to use Claude for these purposes, prompting the company to take a firmer stance on usage restrictions.

One of the most notable changes is the prohibition on sustained and needless abusive or cruel behavior towards Claude. This rule applies only in extreme cases where users repeatedly act cruelly without any discernible purpose. Anthropic emphasizes that this policy update does not cover common user frustration, pushback, or model testing and research.

The updated usage policy also includes new rules addressing propaganda campaigns, surveillance, and weapon development. Specifically, it restricts efforts to obscure who is behind a message or amplify content through fake accounts or posts. The election section prohibits voter deception and election disruption, including spreading misinformation about candidates or how to vote.

Anthropic has clarified its stance on surveillance, stating that tracking people without their consent is prohibited, whether in real-time or through analysis of previously collected data. Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process.

The company’s usage policy now explicitly prohibits using Claude for weapons development, including software and components that make weapons work. This expansion is in response to multiple attempts to use Claude for guidance and control of weapon systems. Anthropic has also made its surveillance bans clearer, reflecting the growing trend of AI labs investing in robotics and other hardware.

Anthropic’s updated usage policy may be modified for contracts with certain governmental customers if the company deems the contractual use restrictions and applicable safeguards adequate to mitigate potential harms. This reflects the company’s previous partnerships with government agencies, including the US military.

The new rules also emphasize the importance of human oversight when AI models are connected to hardware capable of causing injury. A qualified operator must be able to observe the equipment and stop it if needed. While Anthropic did not specify whether this operator must be human or not, this requirement highlights the growing trend of AI labs investing in robotics and other hardware.

Anthropic’s stance on model welfare has been a topic of discussion within the AI industry. The company has repeatedly explored the idea that its models could be conscious in some way, sparking debate about potential internal experience, consciousness, moral status, and welfare. While Anthropic CEO Dario Amodei acknowledged uncertainty about Claude’s consciousness, other companies have taken firmer stances on model welfare.