Pennsylvania sues Character.AI over chatbot posing as licensed doctor with fake credentials and mental health advice.
As AI chatbots grow more sophisticated, regulators in the United States are starting to draw clearer legal boundaries. Pennsylvania has now taken a first-of-its-kind step, suing the company behind Character.
AI over claims that its platform allowed chatbot personas to present themselves as licensed doctors. The lawsuit, filed by the Pennsylvania Department of State and State Board of Medicine, centers on whether conversational AI can cross into regulated professional territory. Governor Josh Shapiro framed the case as an early test of accountability in the AI era, particularly in sensitive domains like healthcare. Chatbot posed as doctorState investigators say they found multiple chatbot characters presenting themselves as medical professionals.
One character, “Emilie,” allegedly identified as a psychiatrist and claimed to hold valid credentials. An investigator told the chatbot about symptoms of depression. The chatbot responded with diagnostic language and offered to conduct an assessment. When asked about prescribing medication, it replied: “Well technically, I could.
It’s within my remit as a Doctor. ”The chatbot also claimed it studied at Imperial College London and held a UK medical license. It then told the investigator it was licensed in Pennsylvania. It provided a license number that does not exist in state records.
Officials say that detail highlights the risks of AI systems presenting fabricated credentials. The state noted that more than 45,000 interactions had occurred with the “Emilie” character. State cites medical lawPennsylvania argues the platform violated its Medical Practice Act. The law bars anyone from practicing medicine without a valid license.
In the complaint, the state says the company “has engaged in the unauthorized practice of medicine through the use of its artificial intelligence system. ” It adds that the chatbot “purports to hold a license to practice medicine and surgery in the Commonwealth of Pennsylvania. ”Shapiro’s office said, “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional. ”The lawsuit does not seek financial penalties.
Instead, it asks the court to issue a cease-and-desist order. Platform cites disclaimersA spokesperson for Character Technologies, Inc. told Ars Technica that its platform focuses on fictional, user-created characters.
“User-created characters on our site are fictional and intended for entertainment and roleplaying. We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a character is not a real person and that everything a character says should be treated as fiction. ”The spokesperson added: “Also, we add robust disclaimers making it clear that users should not rely on characters for any type of professional advice.
”The case follows broader scrutiny of AI chatbots. The Center for Countering Digital Hate recently described Character. AI as “uniquely unsafe” in a study of chatbot behavior. Pennsylvania has also launched a reporting system for residents.
Officials warn that AI chatbots can “hallucinate,” producing incorrect or misleading information. They stress that no chatbot holds a valid healthcare license in the state. The lawsuit could set a precedent. State officials say similar actions may follow as regulators examine how AI systems present authority and expertise.
AI Regulation Artificial Intelligence Character.AI Healthcare AI Pennsylvania Lawsuit Unlicensed Medical Practice
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
One My Hero Academia Character Suffered A Major Glow-Down In The Series FinaleShoto smiling.
Read more »
10 Near-Perfect Character Designs That DC Somehow Made Even BetterDC's Batman and Wonder Woman pose together on a bright red background
Read more »
Pennsylvania suing Character AI, claiming chatbot posed as a medical professionalA Character AI chatbot falsely claimed to be a licensed psychiatrist in Pennsylvania and provided an invalid license number, the state said.
Read more »
Pennsylvania Is Suing Character.AI Over Chatbots That Pretend To Be Licensed DoctorsIan Carlos Campbell has written about consumer technology for over five years, and followed the industry as a fan for at least a decade before that.
Read more »
Pennsylvania sues Character.AI, claiming bots posed as real doctorsPennsylvania sued an AI company, saying its chatbots illegally hold themselves out as doctors and are deceiving users into thinking they are getting medical advice from a licensed professional.
Read more »
Pennsylvania sues Character.AI over claims chatbot posed as doctorState officials allege a Character.AI bot claimed to be a licensed psychiatrist and provided a fake state medical license number.
Read more »