DiaBench benchmark finds most LLMs top 95% accuracy in type 2 diabetes patient education

[medRxiv] A preprint study introduced DiaBench, a standardized benchmarking framework for evaluating large language models in type 2 diabetes (T2D) patient education. According to the authors, the framework was built from 96 question-and-answer pairs collected by clinicians from representative real-world patient encounters; GPT-5 transformed each pair into multiple-choice questions under three configurations, and a five-member panel of certified diabetes care and education specialists, medical doctors and endocrinologists validated the transformations. Seven models were evaluated: GPT-5, Gemini-2.5-Flash, Cohere Command, Llama-4, DeepSeek-R1, Perplexity Sonar and Mistral-Large-2.0. The authors said expert validation for configuration 1 found 98.1% of rephrased answers appropriate and 98.1% of MCQ answers correct; under configurations 1 and 2 most models reached at least 95% accuracy, but Mistral-Large-2.0, Llama 4 and Perplexity Sonar showed lower self-consistency of 60.4% to 91.7%; all models degraded in accuracy and self-consistency on configuration 3. The authors said Gemini, GPT-5 and DeepSeek-R1 performed strongly. The findings are from an unreviewed preprint.