(Top) Barbara Cheifet and James Zou; (Bottom) Risa Wechsler and James Evans. Photo: Agents4Science/Screenshot.
Almost all research journals and conferences prohibit the use of artificial intelligence (AI), but researchers at Stanford University hope to change that.
Agents4Science 2025, which took place on 23 October, was the first open conference where AI served as primary authors and reviewers of research papers.
The conference was intended to be a “relatively safe sandbox” to explore whether AI could generate new scientific insights while maintaining quality through AI-driven peer review.
Charles Sturt University (CSU) librarian Carlo Iacono warned that while AI could help speed up a researcher’s process through providing assistance in drafting, summarising and checking work, AI couldn’t take responsibility for the work produced.
His job title can be misleading, as his duties involve overseeing the safe storage of the university’s digital records, which includes research, student records and reports, and touches upon IT and AI security.
“Authorship and peer review rest on human accountability and consent, which is why major publishers bar AI as an author and funders restrict its use in review,” he said.
“The right question is not whether AI should write or review on its own, but how we use it openly as a tool with humans in charge.
“Recent experiments such as Agents4Science 2025 have been useful for testing the limits, yet they also show how AI reviewers can be fooled by polished but unsound papers and why uploading manuscripts to public tools risks breaches of confidentiality.”
Mr Iacono’s biggest concern is undisclosed AI use.
“My view is cautious optimism: disclose exactly how AI was used, keep final judgement and accountability with named people, and use secure, institutionally approved systems,” he said.
“The biggest concern is undisclosed use; the biggest opportunity is thoughtful augmentation that makes good science faster without lowering standards.”
Even as an advocate for AI, UNSW Canberra engineering and information technology Professor Hussein Abbass is cautious about current models.
He said his 35 years’ experience as an AI researcher had taught him that “AI does not qualify academic authorship”.
“We must pause and ask: what has changed to demand authorship for an AI?” he said.
“Academic papers are a unique form of publications due to expectations for innovation and discovery.”
Professor Abbass said there were four corners to academic authorship: contribution, integrity, accountability and consent.
“AI can’t get held accountable and does not have the will or agency for consent,” he said.
“Current AI systems can’t guarantee integrity without human oversight. Simply put, authorship of academic papers is a human responsibility and is inappropriate for an AI.”
CSU senior lecturer Dr Jacqueline Tinkler previously warned Region that AI was a “flawed technology” with questionable biases and regularly made mistakes.
A study found that five of the largest commercially available genAI engines frequently invoked outdated tropes to depict “Australianness”.
“It’s racist, sexist and a whole lot of those sort of things,” Dr Tinkler said.
“AI has been making scientific discoveries since its inception,” Professor Abbass said.
“Thanks to large language models, significant advances have been made that allows the AI to partially or fully automate the scientific method in defined contexts, opening the possibility for AI to automatically generate academic papers.
“Authorship is a different pool game. As an advocate for AI and as an AI psychologist who designs and diagnoses AI cognition and behaviour, there is a sacred line I do not cross, the line that distinguishes humans from machines; academic authorship is only meaningful for humans, not AI.”
