Development and Preliminary Evaluation of the Patient-Artificial Intelligence Rating Instrument

[medRxiv] Rodman, Lowe, Thomas et al. released a preprint study developing and preliminarily evaluating a patient version of the "Patient–Artificial Intelligence Rating" tool (PAIR), used to assess the quality of health-related conversations between patients and AI. The study conducted an anonymous online cross-sectional survey from June 5 to 14, 2026; 14 adults with experience using large language models to query personal health questions provided a total of 46 complete ratings; of these, 13 provided demographic information, with a median age of 63 years and 8 being women. The tool contains 12 communication quality and patient engagement items (scored 1 to 5) and a separately scored perceived harm item. The paper states that Cronbach α for all ratings was 0.88 (95% confidence interval 0.83 to 0.93), 0.85 when only the first rating was used, and 0.92 when calculated as the mean per participant; the mean quality score was 4.43 (standard deviation 0.45). The intraclass correlation coefficient for the 9 participants who rated multiple times was 0.45, suggesting differences across conversations; of the 46 ratings, 44 (96%) reported no perceived harm. The authors state that the results support further evaluation of the patient-reported perspective, but the sample is small and validity and generalizability need to be verified in larger, more diverse populations…