A general-purpose System One decision model for clinical records

[arXiv] Andrew Zhang, Faisal Mahmood and other authors have released Clinical System One (CS1) on arXiv, a 4-billion-parameter decision model for rapidly making typed judgments from clinical records. The paper states that CS1-4B was trained on 12.4 million question-answer pairs from 3.6 million patients across a multi-hospital health system, can answer multiple questions in a single forward computation without task-specific retraining, and attaches a probability to each answer, allowing uncertain cases to be referred for human review. On 963,408 held-out patients, CS1-4B outperformed open-source decision models with up to 27 billion parameters on 13 of 14 tasks independent of its training teacher's ratings; on 21 external datasets, its overall performance was close to the strongest 27-billion-parameter model in the tests, with inference 6.4 times faster under equal load. The authors state that, when combined with a language model agent, the system screened patients against the enrollment criteria of 17 registered clinical trials, identifying 4 times as many eligible patients among the top 20 candidates as keyword search, while using only about 1/690 of the language model tokens required to read all clinical records one by one.