Multimodal Artificial Intelligence for Cause-of-Death Assignment From Verbal Autopsy Records: A National Benchmark Study in South Africa

[medRxiv] Chu, Wu, McCormick et al. conducted a benchmark study on 4,999 verbal autopsy records of decedents aged 12 and older from a nationally registered death sample in South Africa, using physician-assigned cause of death as the reference. The paper states that all supervised pipelines improved top cause-of-death accuracy by 8.5 to 15.9 percentage points over the current automated method InSilicoVA, and improved cause-specific mortality fraction score accuracy by 3.5 to 7.1 percentage points. A pretrained language model using only narrative text and a machine learning model using only the structured questionnaire performed similarly overall, but the narrative model was more accurate for non-communicable causes of death, while the questionnaire model was more accurate for communicable causes of death. The decision-fusion pipeline further increased top-cause accuracy by 4.2 to 5.4 percentage points, and feature fusion also improved macro-F1 and balanced accuracy, with cause-specific mortality fraction score accuracy reaching 95.7%. Zero-shot large language models performed notably well on rare causes of death. The authors state that the results support incorporating narrative text into automated verbal autopsy systems.