Real-world performance of open-source large language models in diabetes diagnosis

This study evaluated diverse open-source large language models for diagnosing diabetes subtypes and comorbidities from unstructured clinical narratives in a large real-world Chinese cohort of 11,329 adults. The models performed strongly on complex diabetes subtype classification, reaching a peak F1 score of 0.951, while remaining less reliable for more rule-based tasks such as diabetic kidney disease and metabolic syndrome diagnosis. The results suggest that open-source LLMs are valuable clinical co-pilots for complex pattern recognition, with current limitations in procedural diagnostic reasoning.