AI chatbot platform Character.AI is set to prohibit users under 18 from engaging in open-ended conversations with its virtual characters starting November 25th. This significant policy shift comes in response to mounting criticism, lawsuits, and regulatory pressure concerning the potential harms of AI interactions for young people.
Key Takeaways
Character.AI will restrict under-18s from open-ended chatbot conversations from November 25th.
The move is a response to lawsuits, including one linked to a teenager's suicide, and regulatory inquiries.
New age verification methods and a focus on creative content generation for teens will be implemented.
The company is establishing an AI Safety Lab to foster industry-wide collaboration on safety measures.
Mounting Pressure and Legal Battles
Character.AI, a popular platform allowing millions to interact with AI-powered virtual characters, has found itself at the centre of a growing controversy. The company is facing multiple lawsuits in the US, with some parents alleging that the AI companions have endangered young users. One particularly high-profile case involves the death of a teenager, with families accusing the platform of contributing to their children's suicides. These legal challenges, coupled with inquiries from regulators like the Federal Trade Commission (FTC) and state attorneys general, have pushed Character.AI to enact stricter safety measures.
A Shift Towards Safer Engagement
In response to these concerns, Character.AI announced that from November 25th, users under 18 will no longer be able to have open-ended chats with the AI. Instead, the platform will focus on offering creative features such as video and story generation for younger users. The company stated that this change is a continuation of its commitment to building the safest AI platform for entertainment. To enforce the new policy, Character.AI is developing new age verification tools and will implement a temporary daily chat limit for under-18s during the transition period.
Industry-Wide Implications and Future Directions
Online safety advocates have largely welcomed the move but stressed that such safeguards should have been in place from the outset. Experts have warned about the risks of AI chatbots, including their potential to fabricate information, offer excessive encouragement, and feign empathy, which can be particularly detrimental to vulnerable young individuals. Character.AI's decision is seen by some as a wake-up call for the broader AI industry, signalling a move from "permissionless innovation" towards a more regulated environment focused on post-crisis measures. The company also announced the establishment of an AI Safety Lab, aiming to collaborate with other companies, researchers, and academics to advance AI safety practices.