Real-World Performance of Open-Source Large Language Models in Diabetes Diagnosis

This study evaluated how diverse open-source large language models perform on diabetes diagnosis from unstructured clinical text in a real-world Chinese cohort. Across 11,329 adult patients, the models achieved strong performance for complex diabetes subtype classification, but remained less reliable for rule-based diagnoses such as diabetic kidney disease and metabolic syndrome. The results highlight the promise of open-source LLMs as clinical decision-support tools while also underscoring their current limitations in procedural reasoning tasks.