Rhetorical Influence AI Review: Same Paper, Different Score in 4,200 Tests

A growing body of scientific writing is now judged, at least in part, by artificial intelligence, and a new study suggests that the words a researcher chooses can shift an AI reviewer’s verdict even when the underlying data never change. Researchers examining rhetorical influence on AI review found that framing, not evidence, can quietly tilt scores up or down in large language model-based peer review systems, raising fresh questions about how reliable machine judges really are.

The concern isn’t hypothetical. Peer review, the decades-old system where researchers vet each other’s work before publication, is already buckling under a flood of submissions. A recent Frontiers journals survey cited by Ars Technica found that over half of peer reviewers already lean on AI tools somewhere in the review process, often just to keep up with the volume. That reliance is exactly what makes the question of rhetorical bias in AI review so pressing: if machines are already grading papers, and word choice alone can move their scores, the stakes for scientific fairness go up fast.

… Continue reading the full article at the original source below.