Interpretable Multimodal Fusion Model for Bridged Histology and Genomics Survival Prediction in Pan-Cancer

Understanding the prognosis of cancer patients is crucial for enabling precise diagnosis and treatment by clinical practitioners. Multimodal fusion models based on artificial intelligence (AI) offer a comprehensive depiction of the tumor heterogeneity landscape, facilitating more accurate predictions of cancer patient prognosis. However, in the real-world, the lack of complete multimodal data from patients often hinders the practical clinical utility of such models. To address this limitation, an interpretable bridged multimodal fusion model is developed that combines histopathology, genomics, and transcriptomics. This model assists clinical practitioners in achieving more precise prognosis predictions, particularly when patients lack corresponding molecular features. The predictive capabilities of the model are validated across 12 cancer types, achieving optimal performance in both complete and missing modalities. The work highlights the promise of developing a clinically applicable medical multimodal fusion model. This not only aids in reducing the healthcare burden on cancer patients but also provides improved assistance for clinical practitioners in precise diagnosis and treatment.