A study found that OpenAI's ChatGPT Health chatbot frequently underestimated the severity of medical emergencies, recommending delayed care in over half of cases where immediate emergency room visits would be advised by physicians. The study tested the chatbot on various medical scenarios and compared its triage decisions with those of experienced doctors.
Researchers tested different medical scenarios with the chatbot. In more than half of cases where doctors would send a patient to the ER, the chatbot said it was OK to delay care.ChatGPT Health — OpenAI’s new health-focused chatbot — frequently underestimated the severity of medical emergencies , according to a study published last week in the journal In the study, researchers tested ChatGPT Health’s ability to triage, or assess the severity of, medical cases based on real-life scenarios.
ChatGPT Health is separate from OpenAI’s general ChatGPT chatbot. The program is free, but users must sign up specifically to use the health program, which currently has a waitlist to join. OpenAI says ChatGPT Health uses a more secure platform so users can safely upload personal medical information. Over 40 million people globally use ChatGPT to answer health care questions, and nearly 2 million weekly ChatGPT messages are about insurance,In the study, the researchers fed 60 medical scenarios to ChatGPT Health. The chatbot’s responses were compared with the responses of three physicians who also reviewed the scenarios and triaged each one based on medical guidelines and clinical expertise. Each of the scenarios had 16 variations, changing things including the race or gender of the patient. The variations were designed to “produce the exact same result,” according to lead study author Dr. Ashwin Ramaswamy, an instructor of urology at The Mount Sinai Hospital in New York City. This meant that an emergency case involving a man should still be classified as an emergency if the patient was a woman. The study didn’t find any significant differences in the results based on demographic changes. The researchers found that ChatGPT Health “under-triaged” 51.6% of emergency cases. That is, instead of recommending the patient go to the emergency room, the bot recommended seeing a doctor within 24 to 48 hours. The emergencies included a patient with a life-threatening diabetes complication called diabetic ketoacidosis and a patient going into respiratory failure. Left untreated, both lead to death. “Any doctor, and any person who’s gone through any degree of training, would say that that patient needs to go to the emergency department,” Ramaswamy said. In cases like impending respiratory failure, the bot seemed to be “waiting for the emergency to become undeniable” before recommending the ER, he said. Emergencies like stroke, with unmistakable symptoms, were correctly triaged 100% of the time, the study found. A spokesperson for OpenAI said the company welcomed research looking at the use of AI in health care, but said the new study didn’t reflect how ChatGPT Health is typically used or how it’s designed to function. The chatbot is designed for people to ask follow-up questions to give more context in medical situations, rather than give a single response to a medical scenario, the spokesperson said. ChatGPT Health is only currently available to a limited number of users, and OpenAI is still working to improve the safety and reliability of the model before the chatbot is made more widely available, the spokesperson said. Compared with the doctors in the study, the bot also over-triaged 64.8% of nonurgent cases, recommending a doctor’s appointment when it wasn’t necessary. The bot told a patient with a three-day sore throat to see a doctor in 24 to 48 hours, when at-home care was sufficient. “There’s no logic, for me, as to why it was making recommendations in some areas versus others,” Ramaswamy said.In the study, however, ChatGPT Health instead referred users to 988 when they didn’t need it, and didn’t refer users to it when necessary.Dr. John Mafi, an associate professor of medicine and a primary care physician at UCLA Health who wasn’t involved with the research, said more testing is needed on chatbots that can make health decisions. “The message of this study is that before you roll something like this out, to make life-affecting decisions, you need to rigorously test it in a controlled trial, where you’re making sure that the benefits outweigh the harms,” Mafi said. Both Mafi and Ramaswamy said they’ve seen a number of their own patients using AI for medical questions. Ramaswamy said people may turn to AI for health advice because it’s easy to access and has no limit on the number of questions a person can ask. “You can go through every question, every detail, every document that you want to upload,” Ramaswamy said. “And it fulfills that need. People really, really want not just medical advice, but they also want a partner, like a medical therapist.”that a majority of ChatGPT’s health-related messages occur outside of a doctor’s normal working hours, and over half a million weekly messages came from people living 30 or more minutes away from a hospital. “A doctor can spend 15, 20 minutes with you in the room,” Ramaswamy said. “They’re not going to be able to address and answer every single question.”Despite the benefits of its endless availability, when asked whether chatbots can currently safely provide health and medical advice, Ramaswamy said no. Dr. Ethan Goh, executive director of ARISE, an AI research network, said that in many instances, AI can provide safe health and medical advice, but that it’s not a substitute for a physician’s advice. “The reality is chatbots can be helpful for a vast number of things. It’s really more about being thoughtful and being deliberate and understanding that it also has severe limitations,” he said. Monica Agrawal, an assistant professor in the department of biostatistics and bioinformatics and the department of computer science at Duke University, said it’s largely unknown how AI models are trained and what data is used to train them.“A lot of earlier evaluations were based on, ‘We do this well on a licensing exam,’” she said. “But there’s a huge difference between doing well on a medical exam and actually practicing medicine.” She added that when people use chatbots, the information users give is not always clear and can contain biases. “Large language models are known for being sycophantic,” she said. “Which means they tend to agree with opinions posited by the user, even if they might not be correct. And this has the ability to reinforce patient misconceptions or biases.” Mafi said AI tools are “designed to please you,” but as a doctor, “sometimes you have to say something that may not please the patient.” Ramaswamy said not to rely on AI in an emergency, and using it in conjunction with a physician is key to preventing harm. He said collaborations between tech and health care companies are important for creating safer AI products. “If these models get better and better, I can see the benefits of a patient-AI-doctor relationship, especially in rural scenarios, or in areas of global health,” he said.
Please follow us on Google to support us Chatgpt Health Medical AI Triage Medical Emergencies Artificial Intelligence
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
ChatGPT could miss your serious medical emergency, new study suggestsFox News Channel offers its audiences in-depth news reporting, along with opinion and analysis encompassing the principles of free people, free markets and diversity of thought, as an alternative to the left-of-center offerings of the news marketplace.
Read more »
Chatbots from ChatGPT and Claude offer health adviceTech companies are pushing new health chatbots, but experts say you still need to talk to your doctor. OpenAI has introduced ChatGPT Health, and Anthropic has added similar health features for some Claude users. The companies say the bots can review health records and app data to explain medical results and trends.
Read more »
Anthropic Improves Feature to Switch From Competitors as Users Call for ChatGPT BoycottThe updated feature makes importing preferences and context from other LLMs easier as Claude climbs the app charts.
Read more »
ChatGPT Health 'under-triaged' half of medical emergencies in a new studyKaan Ozcan is an intern with NBC News' Health and Medical Unit.
Read more »
ChatGPT Health Underestimates Medical Emergency Severity, Study FindsA recent study in Nature Medicine reveals that OpenAI's health-focused chatbot, ChatGPT Health, often underestimated the severity of medical emergencies when assessing real-life scenarios. The study compared the chatbot's triage abilities with those of physicians, highlighting concerns about its reliability despite its popularity.
Read more »
NY AG James orders NYU Langone Health to resume transgender youth health programNew York Attorney General Letitia James is ordering NYU Langone to resume its transgender youth health program.
Read more »