AI and Peer Review 2025
Insights from the global community
Conclusion
Many academic publishers have introduced increasingly nuanced policies that reflect the delicate balance between efficiency, innovation and research integrity. Most publisher policies prohibit the use of consumer-facing generative AI tools to conduct analysis or evaluation of submitted manuscripts or to upload all or part of submitted manuscripts to AI tools, due to the associated privacy, confidentiality and integrity concerns. This stance aligns with feedback from the researcher community (both in this survey and in others) which indicates widespread discomfort with AI-generated or AI-augmented reviews among authors. However, the responses also highlight that there are opportunities for generative AI to ease the workload of peer reviewers, with a sizeable proportion of the community indicating that they feel peer review tools should be used in peer review for tasks like language and grammar editing. This is where many current publisher policies fall short.
The challenge for publishers is to reconcile these two viewpoints. One solution could be the development of AI tools that operate within peer review software and systems to provide support for reviewers without posing security or integrity risks. These tools could be monitored by editors and should be designed to support, rather than replace, human judgment. If implemented effectively, such tools would not only address ethical concerns but also mitigate risks around confidentiality and data privacy; particularly the issue of reviewers uploading manuscripts to third-party generative AI platforms.
Furthermore, publishers may benefit from being more explicit and transparent about why LLMs and AI chatbots are not suitable tools for fully authoring peer review reports. As many respondents pointed out, while the outputs of these tools might often appear polished and give the illusion of useful scientific critique, they usually lack depth because LLMs are not subject experts and are unable to engage in higher-level reasoning or critique. A minority of respondents also demonstrated misunderstandings about how these models function and what their limitations are, underscoring the need for clearer communication and education around the capabilities and boundaries of generative AI.
More than one-third of respondents admitted to already using generative AI to write or augment peer review reports, even though this is against the policies of a lot of publishers. This creates a real challenge for both publishers and authors, as it is often difficult to tell whether a report has been written entirely by AI or simply re-written using a chatbot for grammar and flow. While there are some recognizable hallmarks, the boundary between human and AI input is becoming increasingly hard to define. What is clear is that current policies are not preventing widespread use, and this lack of transparency is problematic for everyone involved in the peer review process. Reviewers are turning to these tools to reduce their workloads. If publishers want to maintain trust and uphold standards, something needs to change in how AI use is disclosed and managed.
As generative AI tools continue to change, publishers must respond nimbly. It is essential to recognize that attitudes towards generative AI may vary across disciplines, career stages, and gender groups. Any policies or tools must be flexible enough to accommodate this diversity of viewpoints, while maintaining the integrity and trust that underpin the peer review process.
Recommendations
- Harmonize policies across publishers.
- Invest in AI tools that operate within peer review software and systems to provide support for reviewers without posing security or integrity risks.
- Prioritize clear communication and education around the capabilities and boundaries of generative AI.
- Ensure that any policies or tools are flexible enough to accommodate diverse viewpoints and maintain the integrity and trust that underpins the peer review process.