The most recent version of ChatGPT, an AI chatbot developed for language interpretation and response generation, has successfully passed a radiology board-style exam, demonstrating both its potential and limitations, according to research studies published in the Radiological Society of North Americ
The latest version of ChatGPT passed a radiology board-style exam, highlighting the potential of large language models but also revealing limitations that hinder reliability, according to two new research studies published inChatGPT is an artificial intelligence chatbot that uses a deep learning model to recognize patterns and relationships between words in its vast training data to generate human-like responses based on a prompt.
To assess its performance on radiology board exam questions and explore strengths and limitations, Dr. Bhayana and colleagues first tested ChatGPT based on GPT-3.5, currently the most commonly used version. The researchers used 150 multiple-choice questions designed to match the style, content and difficulty of the Canadian Royal College and American Board of Radiology exams.
The researchers found that ChatGPT based on GPT-3.5 answered 69% of questions correctly , near the passing grade of 70% used by the Royal College in Canada. The model performed relatively well on questions requiring lower-order thinking , but struggled with questions involving higher-order thinking . More specifically, it struggled with higher-order questions involving description of imaging findings , calculation and classification , and application of concepts .
The findings suggest that GPT-4’s claimed improved advanced reasoning capabilities translate to enhanced performance in a radiology context. They also suggest improved contextual understanding of radiology-specific terminology, including imaging descriptions, which is critical to enable future downstream applications.
“We were initially surprised by ChatGPT’s accurate and confident answers to some challenging radiology questions, but then equally surprised by some very illogical and inaccurate assertions,” Dr. Bhayana said. “Of course, given how these models work, the inaccurate responses should not be particularly surprising.”
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
College instructor put on blast for accusing students of using ChatGPTTexas A&M University–Commerce says it is investigating after a screenshot of an instructor's email — in which he accused students of having used AI on their final assignments — went viral.
Read more »
Nebraska high school student goes off-script after getting ChatGPT-written graduation speech approvedKenny Morales, a student at Grand Island Senior High School in Nebraska, says he used AI to draft a decoy speech that was approved by officials, a report says.
Read more »
Gartner Identifies Six ChatGPT Risks Legal and Compliance Leaders Must EvaluateGartnerLegal has identified 6 ChatGPT risks legal and compliance leaders need to evaluate. Read the press release here: Gartner_inc
Read more »
OpenAI launches free ChatGPT app for iOSAn Android version is on the way.
Read more »
ChatGPT’s Creator Buddies Up to Congress | Future TechOpenAI CEO Sam Altman showed support for the formation of a regulatory body for AI in front of Congress.
Read more »
ChatGPT Now Has an iPhone AppSix months after OpenAI’s ChatGPT launched on the web and set off an AI arms race, its iPhone app is here.
Read more »