LLMs detect methodological errors missed in human peer review, study finds, with top score at 99%

[medRxiv] A preprint study reports that in an earlier experiment with errors inserted into three randomized clinical trial reports, untrained human reviewers achieved a mean major-error detection score of 27%, while 15 large language models, including four open-weight models, scored between 53% and 96%. To reduce the risk of memorization, the researchers repeated the test on three clinical trials published in 2026; scores ranged from 51% to 99%, and from 66% to 87% for the open-weight models. According to the study, the LLMs still flagged 2% to 29% of the error types in control reports without inserted errors, most often conclusions not supported by the results and missing reasons why screened patients were not enrolled. Mean review times across the 12 models ranged from 1.4 to 16.3 minutes. The study said Opus 5.5 achieved the highest major-error detection score at 99%, while GPT-6.1 Sol and GPT-6 Luna scored 92% and 87%, respectively. The authors said the tests provide a benchmark for specialized review models and suggest that relying on human peer review alone may leave avoidable methodological errors undetected.