Multimodal Deep Learning for Disease Diagnosis and Risk Stratification: Integrating Genomic, Clinical, and Imaging Data

Tata Balaji; Gonuguntla Vamsi Krishna; Polukonda Ravi Kumar; Mekhala Sri Devi Sameera; Vemu Suma Avani; Ganugapati Naga Sowjanya

Authors

Tata Balaji Department of Electronics and Communication Engineering, PVP Siddhartha Institute of Technology, Vijayawada, AP, India https://orcid.org/0000-0002-1237-6457
Gonuguntla Vamsi Krishna Department of Electronics and Instrumentation Engineering, VNR Vignana Jyothi Institute of Engineering and Technology, Hyderabad, Telangana, India https://orcid.org/0000-0003-0239-4632
Polukonda Ravi Kumar Department of Electronics and Communication Engineering, Shri Vishnu Engineering College for Women, Bhimavaram, AP, India. https://orcid.org/0000-0003-4052-1196
Mekhala Sri Devi Sameera Department of Computer Science and Engineering, Dhanekula Institue of Engineering & Technology, Vijayawada, A.P, India. https://orcid.org/0009-0005-6968-7263
Vemu Suma Avani Department of Data Science & Engg.Vijaya Institute of Technology for women, Enikepadu, Vijayawada, A.P, India https://orcid.org/0000-0001-7296-2768
Ganugapati Naga Sowjanya Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur, AP, India. https://orcid.org/0009-0008-9613-9109

Keywords:

Personalized Healthcare, Multimodal Deep Learning, Genomics, Clinical Data, Medical Imaging, Precision Medicine

Abstract

Personalized healthcare depends on the smart combination of heterogeneous biomedical information, including genomic sequences, clinical records, and medical imaging, so that it can be predictable with precision and interpretation. To accomplish this, the current study suggests a Hierarchy Attention Fusion based Multimodal Deep Learning (HAF-MDL) framework which improves the diagnostic accuracy and interpretability by intra- and inter-modality attention and Bayesian uncertainty measurement. In contrast to the conventional fusion methods, HAF-MDL learns the modality-relevant dynamically, avoiding uncertainty in heterogeneous patient data. To make the model clinical, it was trained and evaluated using a semi-synthetic dataset of 1,440 patient profiles in statistical agreement with real biomedical repositories TCGA (oncology), MIMIC-IV (clinical), and ADNI (neurology) to make the model clinically realistic. The Kolmogorov Smirnov (Ks) tests (p > 0.05) validation was also performed to ensure that the generated distributions were statistically consistent with real data in the world, which improved the reproducibility. The HAF-MDL framework proposed reached an accuracy of 94.8% and AUC of 0.964, which is higher than the unimodal and conventional fusion models. These results show that the suggested multimodal integration plan has great benefits in terms of the disease diagnosis and risk stratification and provides interpretability and reliability, generating a repeatable pathway to precision medicine.