A system of five models helps peer reviewers to write more constructive comments, but it is not yet known whether this strengthens the papers that are being reviewed. A system of five models helps peer reviewers to write more constructive comments, but it is not yet known whether this strengthens the papers that are being reviewed.
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser . In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
An artificial-intelligence coach can help peer reviewers to provide more constructive and less toxic feedback, according to a new study, a computer scientist at Stanford University in California, and his colleagues set out to assess whether large language models could help to address a common complaint about: feedback often lacks thoroughness or strikes the wrong tone. At the 2023 Association for Computational Linguistics annual meeting in Toronto, Canada, for example, authors of conference papers flagged 12.9% of reviews as being poor quality. That’s mainly because the reviews were vague, says Zou, with broad, simple comments such as “not novel”. Reviews can also, rarely,, with comments such as “these authors don’t know what they’re talking about”, says Zou. Others make factual errors, for example criticizing work for omitting an analysis when that analysis is, in fact, there.Zou and his colleagues gathered about a dozen reviews that were vague, unprofessional or incorrect, along with what they considered to be appropriate feedback about those reviews. They fed that curated data to an LLM to help refine its responses and used this to develop a Review Feedback Agent, which uses a total of five LLMs to collaborate and check each others’ work. The team put their AI tool to work in the lead up to the 2025 International Conference on Learning Representations in Singapore. This major AI conference has attracted more than 10,000 submissions for the past few years. Each paper is reviewed by 3–4 people and around 30% are accepted.The group randomly selected around 20,000 already-written reviews, used the Review Feedback Agent to evaluate them, and sent the reviewers the AI tool’s feedback. Most of the time the AI system suggested ways that reviewers could be more specific and constructive, frequently using the phrase “to make this feedback more actionable …”.Access the most recent journalism from Nature's award-winning teamHangzhou, ChinaPost-doc in Organic Electronics at Umeå University, Sweden A two-year postdoctoral position in Organic Electronics focused on the development of a novel light source, a light-emitting electrochemical cell, LECTwo postdoctoral scholarships in AI-driven marine biology
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
2026 Winter Olympics live updates: U.S. faces Canada in hockey gold medal gameThis is additional taxonomy that helps us with analytics
Read more »
2026 Olympics closing ceremony live updates: How to watch, time, channel and moreThis is additional taxonomy that helps us with analytics
Read more »
Armed man shot and killed at Mar-a-LagoThis is additional taxonomy that helps us with analytics
Read more »
Seinfeld: 10 Classic Episodes That Still Outshine Modern SitcomsKramer helps Jerry put on the puffy shirt in Seinfeld
Read more »
Photo shows shotgun, gas can carried by man shot and killed at Mar-a-LagoThis is additional taxonomy that helps us with analytics
Read more »
Winter storm live updates: Northeast prepares for historic blizzardThis is additional taxonomy that helps us with analytics
Read more »