New study shows AI large language models (LLMs) outperform human neuroscientists in predicting neuroscience study outcomes.
At the crossroads of psychology and biology is the inherently complex life sciences field of neuroscience, the study of the brain anddemonstrates how AI large language models outperform human neuroscientists in predicting neuroscience study outcomes.
“We foresee a future in which LLMs serve as forward-looking generative models of the scientific literature,” wrote University College London Psychology and Language Sciences postdoctoral research fellow Xiaoliang Luo, PhD, and cognitive and decision sciences professor Bradley Love, PhD, and their wide consortium of research colleagues affiliated with multiple institutions from around the world. LLMs are immense AI deep learning models that are pre-trained on massive amounts of data that are able to generate and process human language. The scientists for this study set out to evaluate whether or not LLMs could tackle complex tasks that are difficult for human neuroscientists to perform. "LLMs’ predictions are informed by a vast scientific literature that no human could read in their lifetime," wrote the scientists. Predicting the outcome of neuroscience studies is a daunting task for neuroscientists. Factors such as the diversity of neuroscience research methods. Neuroscience research methods include brain imaging such as functional magnetic resonance imaging , electroencephalography , magnetoencephalography , and positron emission tomography , organoids derived from pluripotent stem cells, and pharmacological interventions, to name just a few. Other contributing factors that make predicting neuroscience research results challenging for experts include the breadth of multiple levels that range from molecular biology to behavior, the massive volume of pertinent science publications that could be in the thousands, the intricacy and variety of analytical techniques, and the study’sThe researchers developed a forward-looking benchmark for neuroscience called BrainBench to quantify and compare the ability of various general-purpose LLMs versus 171 human neuroscience experts who passed a screening test to predict neuroscience research outcomes. Several different versions of Llama, Galactica, Falcon, and Mistral comprised a total of 15 LLMs that were evaluated in this study. The test cases included the five neuroscience areas of behavioral/cognitive, systems/circuits, neurobiology of disease, development/plasticity/repair, and cellular/molecular. The results were clear—each LLM beat human neuroscience experts by a wide margin. The LLMs average accuracy of 81.4 percent far exceeded the 63.4 percent average of human experts. Next, the scientists created a new LLM called BrainGPT by fine-tuning an existing version of Mistral and training data from twenty years of neuroscience publications from a hundred journals published during 2002-2022. BrainGPT had an 86 percent accuracy in predicting neuroscience study results, which was a three percent gain from the general-purpose version of Mistral. "LLMs can be part of larger systems that assist researchers in determining the best experiment to conduct next," the researchers wrote. The ability to predict results of neuroscience research in advance can help guide neuroscientists to optimize limited resources such as time and money, enable timely adjustments based on probable outcomes, and augment our understanding of the brain and central nervous system that may lead to better treatments and health interventions. This proof of concept is not limited to neuroscience. According to the scientists, none of their methods used was specific just to neuroscience and can be applied more broadly to other knowledge-intensive domains in the future.When we fall prey to perfectionism, we think we’re honorably aspiring to be our very best, but often we’re really just setting ourselves up for failure, as perfection is impossible and its pursuit inevitably backfires.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
ChatGPT Can Tell You What Scientists Are Doing With LLMsConfused about LLM architectures? Ask a model. They’ll tell you.
Read more »
When training medical LLMs, specialization may not always be better.General AI models rival specialized ones in most medical tasks, proving that optimized prompts and strategic use may outpace costly domain-specific training.
Read more »
Stretching From LLMs To LGMs: Intelligence And The Amazing Promise Of Large Geospatial ModelsDr. Lance B. Eliot is a world-renowned AI scientist and consultant with over 8.1+ million amassed views of his AI columns and been featured on CBS 60 Minutes. As a CIO/CTO seasoned executive and high-tech entrepreneur, he combines practical industry experience with deep academic research.
Read more »
LLMs: The timeless role of the scribe meets the transformative power of AI.LLMs are modern scribes—dynamic collaborators that transform scattered ideas into refined expression, reshaping how we think, create, and solve problems.
Read more »
How AI and LLMs are changing the way we "think" about being smart.Technology Quotient (TQ) redefines intelligence, showing how we think, create, and learn with AI as a collaborative partner.
Read more »
Orange Partners With OpenAI To Make LLMs More Inclusive In AfricaMeghan McCormick is a Ghana-based entrepreneur covering women building high-impact organizations across the African continent. She's written about Ghana's successful 'Year of Return', entrepreneurs moving to Africa to launch businesses, and women leading the continent's burgeoning tech scene.
Read more »